concepts, tools, reviews, inspections & walkthroughs; p ... · cs621 – software quality...
TRANSCRIPT
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 1 hcr:innovationcse@gg
UNIT IV SOFTWARE PROCESSES & TESTING Software Process - Definition and implementation; internal Auditing and Assessments; Software testing -
Concepts, Tools, Reviews, Inspections & Walkthroughs; P-CMM.
SOFTWARE PROCESS - DEFINITION AND
IMPLEMENTATION
The software process
A structured set of activities required to develop a software system.
Many different software processes but all involve:
o Specification – defining what the system should do;
o Design and implementation – defining the organization of the system and implementing the system;
o Validation – checking that it does what the customer wants;
o Evolution – changing the system in response to changing customer needs.
A software process model is an abstract representation of a process. It presents a description of a process
from some particular perspective
Software process descriptions
activities in these processes such as specifying a data model, designing a user interface, etc. and the
ordering of these activities
Process descriptions may also include:
o Products, which are the outcomes of a process activity;
o Roles, which reflect the responsibilities of the people involved in the process;
o Pre- and post-conditions, which are statements that are true before and after a process activity has
been enacted or a product produced
Software process models
The waterfall model
o Plan-driven model. Separate and distinct phases of specification and development.
Incremental development
o Specification, development and validation are interleaved. May be plan-driven or agile.
Reuse-oriented software engineering
o The system is assembled from existing components. May be plan-driven or agile.
In practice, most large systems are developed using a process that incorporates elements from all of these
models
Waterfall Model
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 2 hcr:innovationcse@gg
There are separate identified phases in the waterfall model:
o Requirements analysis and definition
o System and software design
o Implementation and unit testing
o Integration and system testing
o Operation and maintenance
The main drawback of the waterfall model is the difficulty of accommodating change after the process is
underway. In principle, a phase has to be complete before moving onto the next phase
Problems
Inflexible partitioning of the project into distinct stages makes it difficult to respond to changing customer
requirements.
o Therefore, this model is only appropriate when the requirements are well-understood and changes will
be fairly limited during the design process.
o Few business systems have stable requirements.
The waterfall model is mostly used for large systems engineering projects where a system is developed at
several sites.
o In those circumstances, the plan-driven nature of the waterfall model helps coordinate the work
Incremental development
Incremental development benefits
The cost of accommodating changing customer requirements is reduced.
o The amount of analysis and documentation that has to be redone is much less than is required with the
waterfall model.
It is easier to get customer feedback on the development work that has been done.
o Customers can comment on demonstrations of the software and see how much has been implemented.
More rapid delivery and deployment of useful software to the customer is possible.
o Customers are able to use and gain value from the software earlier than is possible with a waterfall
process
Incremental development problems
The process is not visible.
o Managers need regular deliverables to measure progress. If systems are developed quickly, it is not
cost-effective to produce documents that reflect every version of the system.
System structure tends to degrade as new increments are added.
o Unless time and money is spent on refactoring to improve the software, regular change tends to corrupt
its structure. Incorporating further software changes becomes increasingly difficult and costly
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 3 hcr:innovationcse@gg
Reuse-oriented software engineering
Based on systematic reuse where systems are integrated from existing components or COTS (Commercial-
off-the-shelf) systems.
Process stages
o Component analysis;
o Requirements modification;
o System design with reuse;
o Development and integration.
Reuse is now the standard approach for building many types of business system
Types of software component in reuse
Web services that are developed according to service standards and which are available for remote
invocation.
Collections of objects that are developed as a package to be integrated with a component framework such
as .NET or J2EE.
Stand-alone software systems (COTS) that are configured for use in a particular environment
Process activities
Real software processes are inter-leaved sequences of technical, collaborative and managerial activities
with the overall goal of specifying, designing, implementing and testing a software system.
The four basic process activities of
o specification,
o development,
o validation and
o evolution
are organized differently in different development processes.
In the waterfall model, they are organized in sequence
in incremental development they are inter-leaved
Software specification
The requirements engineering process
The process of establishing what services are required and the constraints on the system’s operation and
development.
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 4 hcr:innovationcse@gg
Requirements engineering process
o Feasibility study
Is it technically and financially feasible to build the system?
o Requirements elicitation and analysis
What do the system stakeholders require or expect from the system?
o Requirements specification
Defining the requirements in detail
o Requirements validation
Checking the validity of the requirements
Software design and implementation
The process of converting the system specification into an executable system.
Software design
o Design a software structure that realises the specification;
Implementation
o Translate this structure into an executable program;
The activities of design and implementation are closely related and may be inter-leaved
A general model of the design process
Design activities
Architectural design, where you identify the overall structure of the system, the principal components
(sometimes called sub-systems or modules), their relationships and how they are distributed.
Interface design, where you define the interfaces between system components.
Component design, where you take each system component and design how it will operate.
Database design, where you design the system data structures and how these are to be represented in a
database
Software validation
Verification and validation (V & V) is intended to show that a system conforms to its specification and meets
the requirements of the system customer.
Involves checking and review processes and system testing.
System testing involves executing the system with test cases that are derived from the specification of the
real data to be processed by the system.
Testing is the most commonly used V & V activity
Stages of testing
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 5 hcr:innovationcse@gg
Development or component testing
o Individual components are tested independently;
o Components may be functions or objects or coherent groupings of these entities.
System testing
o Testing of the system as a whole. Testing of emergent properties is particularly important.
Acceptance testing
o Testing with customer data to check that the system meets the customer’s needs
Testing phases in a plan-driven software process (V Model)
Software evolution
Software is inherently flexible and can change.
As requirements change through changing business circumstances, the software that supports the
business must also evolve and change.
Although there has been a demarcation between development and evolution (maintenance) this is
increasingly irrelevant as fewer and fewer systems are completely new
System evolution
Summary
Software processes are the activities involved in producing a software system.
Software process models are abstract representations of these processes.
General process models describe the organization of software processes.
Examples of these general models include the ‘waterfall’ model, incremental development, and reuse-
oriented development
Requirements engineering is the process of developing a software specification.
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 6 hcr:innovationcse@gg
Design and implementation processes are concerned with transforming a requirements specification into an
executable software system.
Software validation is the process of checking that the system conforms to its specification and that it meets
the real needs of the users of the system.
Software evolution takes place when you change existing software systems to meet new requirements.
The software must evolve to remain useful
Coping with change
Change is inevitable in all large software projects.
o Business changes lead to new and changed system requirements
o New technologies open up new possibilities for improving implementations
o Changing platforms require application changes
Change leads to rework so the costs of change include both rework (e.g. re-analysing requirements) as well
as the costs of implementing new functionality
Reducing the costs of rework
Change avoidance, where the software process includes activities that can anticipate possible changes
before significant rework is required.
o For example, a prototype system may be developed to show some key features of the system to
customers.
Change tolerance, where the process is designed so that changes can be accommodated at relatively low
cost
Software prototyping
A prototype is an initial version of a system used to demonstrate concepts and try out design options.
A prototype can be used in:
o The requirements engineering process to help with requirements elicitation and validation;
o In design processes to explore options and develop a UI design;
o In the testing process to run back-to-back tests.
Benefits of prototyping
Improved system usability.
A closer match to users’ real needs.
Improved design quality.
Improved maintainability.
Reduced development effort.
The process of prototype development
Prototype development
May be based on rapid prototyping languages or tools
May involve leaving out functionality
Prototype should focus on areas of the product that are not well-understood;
Error checking and recovery may not be included in the prototype;
Focus on functional rather than non-functional requirements such as reliability and security
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 7 hcr:innovationcse@gg
Incremental delivery
Rather than deliver the system as a single delivery, the development and delivery is broken down into
increments with each increment delivering part of the required functionality.
User requirements are prioritised and the highest priority requirements are included in early increments.
Once the development of an increment is started, the requirements are frozen though requirements for later
increments can continue to evolve
Incremental development and delivery
Incremental development
o Develop the system in increments and evaluate each increment before proceeding to the development
of the next increment;
o Normal approach used in agile methods;
o Evaluation done by user/customer proxy.
Incremental delivery
o Deploy an increment for use by end-users;
o More realistic evaluation about practical use of software;
o Difficult to implement for replacement systems as increments have less functionality than the system
being replaced
Incremental delivery
Incremental delivery advantages
Customer value can be delivered with each increment so system functionality is available earlier.
Early increments act as a prototype to help elicit requirements for later increments.
Lower risk of overall project failure.
The highest priority system services tend to receive the most testing
Incremental delivery problems
Most systems require a set of basic facilities that are used by different parts of the system.
o As requirements are not defined in detail until an increment is to be implemented, it can be hard to
identify common facilities that are needed by all increments.
The essence of iterative processes is that the specification is developed in conjunction with the software
Boehm’s spiral model of the software process
Process is represented as a spiral rather than as a sequence of activities with backtracking.
Each loop in the spiral represents a phase in the process.
No fixed phases such as specification or design - loops in the spiral are chosen depending on what is
required.
Risks are explicitly assessed and resolved throughout the process.
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 8 hcr:innovationcse@gg
The Rational Unified Process
A modern generic process derived from the work on the UML and associated process.
Brings together aspects of the 3 generic process models discussed previously.
Normally described from 3 perspectives
o A dynamic perspective that shows phases over time;
o A static perspective that shows process activities;
o A practive perspective that suggests good practice
RUP good practice
Visually model software: Use graphical UML models to present static and dynamic views of the software.
Verify software quality: Ensure that the software meet’s organizational quality standards.
Control changes to software
Manage software changes using a change management system and configuration management tools
Summary
Processes should include activities to cope with change.
o This may involve a prototyping phase that helps avoid poor decisions on requirements and design.
Processes may be structured for iterative development and delivery so that changes may be made without
disrupting the system as a whole.
The Rational Unified Process is a modern generic process model that is organized into phases (inception,
elaboration, construction and transition) but separates activities (requirements, analysis and design, etc.)
from these phases
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 9 hcr:innovationcse@gg
INTERNAL AUDITING AND ASSESSMENTS TBD
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 10 hcr:innovationcse@gg
SOFTWARE TESTING CONCEPTS Software Testing
Once source code has been generated, software must be tested to uncover (and correct) as many errors as
possible before delivery to the customer
software testing techniques provide systematic guidance for designing tests that
o (1) exercise the internal logic of software components,
o (2) exercise the input and output domains of the program to uncover errors in program function,
behavior. and performance
Done by software engineer and testing specialists
Why it is important
Reviews and other SQA activities can and do uncover errors, but they are not sufficient.
Every time the program is executed, the customer tests it!
Therefore, you have to execute the program before it gets to the customer with the specific intent of finding
and removing all errors.
In order to find the highest possible number of errors, tests must be conducted systematically and test
cases must be designed using disciplined techniques.
Software is tested from two different perspectives:
Internal program logic is exercised using “white box” test case design techniques.
Software requirements are exercised using “black box” test case design techniques.
intent is to find the maximum number of errors with the minimum amount of effort and time
Testing Principles
All tests should be traceable to customer requirements
Tests should be planned long before testing begins
The Pareto principle applies to software testing
Testing should begin “in the small” and progress toward testing “in the large.”
Exhaustive testing is not possible
To be most effective, testing should be conducted by an independent third party
Software Testability
how easily [a computer program] can be tested
Set of characteristics that lead to testable software
Operability. "The better it works, the more efficiently it can be tested."
Observability. "What you see is what you test."
Controllability. "The better we can control the software, testing can be automated and optimized."
Decomposability. "controlling the scope, can more quickly isolate problems & perform smarter retesting."
Simplicity. "The less there is to test, the more quickly we can test it."
o Functional, Structural, Code
Stability. "The fewer the changes, the fewer the disruptions to testing."
Understandability. "The more information we have, the smarter we will test."
Attributes of a “good” test
has a high probability of finding an error
not redundant
best of breed
neither too simple nor too complex
Test Case Design
black-box tests are used to demonstrate that software functions are operational, that input is properly
accepted and output is correctly produced, and that the integrity of external information (e.g., a database) is
maintained.
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 11 hcr:innovationcse@gg
A black-box test examines some fundamental aspect of a system with little regard for the internal logical
structure of the software.
White-box testing of software is predicated on close examination of procedural detail. Logical paths through
the software are tested by providing test cases that exercise specific sets of conditions and/or loops.
The "status of the program" may be examined at various points to determine if the expected or asserted
status corresponds to the actual status.
White-Box Testing
sometimes called glass-box testing,
is a test case design method that uses the control structure of the procedural design to derive test cases.
Using white-box testing methods, the software engineer can derive test cases that
o (1) guarantee that all independent paths within a module have been exercised at least once,
o (2) exercise all logical decisions on their true and false sides,
o (3) execute all loops at their boundaries and within their operational bounds, and
o (4) exercise internal data structures to ensure their validity.
Basis Path Testing
Basis path testing is a white-box testing technique
enables the test case designer to derive a logical complexity measure of a procedural design and use this
measure as a guide for defining a basis set of execution paths
Flow Graph Notation
Cyclomatic Complexity
Cyclomatic complexity is a software metric that provides a quantitative measure of the logical complexity of
a program.
When used in the context of the basis path testing method, the value computed for cyclomatic complexity
defines the number of independent paths in the basis set of a program and provides us with an upper
bound for the number of tests that must be conducted to ensure that all statements have been executed at
least once
Deriving Test Cases
Using the design or code as a foundation, draw a corresponding flow graph
Determine the cyclomatic complexity of the resultant flow graph.
Determine a basis set of linearly independent paths
Prepare test cases that will force execution of each path in the basis set
Graph Matrices
A graph matrix is a square matrix whose size (i.e., number of rows and columns) is equal to the number of
nodes on the flow graph.
Each row and column corresponds to an identified node, and matrix entries correspond to connections (an
edge) between nodes.
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 12 hcr:innovationcse@gg
Control Structure Testing
Condition Testing
Condition testing is a test case design method that exercises the logical conditions contained in a program
module.
Data Flow Testing
The data flow testing method selects test paths of a program according to the locations of definitions and
uses of variables in the program
Loop Testing
Loop testing is a white-box testing technique that focuses exclusively on the validity of loop constructs
Four different classes of loops can be defined:
o simple loops,
o concatenated loops,
o nested loops, and
o unstructured loops
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 13 hcr:innovationcse@gg
Black-Box Testing
also called behavioral testing,
focuses on the functional requirements of the software.
enables the software engineer to derive sets of input conditions that will fully exercise all functional
requirements for a program.
Black-box testing is not an alternative to white-box techniques.
it is a complementary approach that is likely to uncover a different class of errors than white-box methods.
Graph-Based Testing Methods
Equivalence Partitioning
Equivalence partitioning is a black-box testing method that divides the input domain of a program into
classes of data from which test cases can be derived
Equivalence classes may be defined according to the following guidelines:
o 1. If an input condition specifies a range, one valid and two invalid equivalence
classes are defined.
o 2. If an input condition requires a specific value, one valid and two invalid
equivalence classes are defined.
o 3. If an input condition specifies a member of a set, one valid and one invalid
equivalence class are defined.
o 4. If an input condition is Boolean, one valid and one invalid class are defined.
Boundary Value Analysis
Boundary value analysis is a test case design technique that complements equivalence partitioning.
Rather than selecting any element of an equivalence class, BVA leads to the selection of test cases at the
"edges" of the class
Comparison Testing
independent versions of software be developed for critical applications,even when only a single version will
be used in the delivered computer-based system
Orthogonal Array Testing
can be applied to problems in which the input domain
is relatively small but too large to accommodate exhaustive testing
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 14 hcr:innovationcse@gg
useful in finding errors associated with region faults—an error category associated with faulty logic within a
software component
Testing For Specialized Environments, Architectures, And Applications
Testing GUIs
o testing should be approached using automated tools
Testing of Client/Server Architectures
Testing Documentation and Help Facilities
Testing for Real-Time Systems
o The time-dependent, asynchronous nature of many real-time applications adds a new and potentially
difficult element to the testing mix—time.
o Not only does the test case designer have to consider white- and black-box test cases but also event
handling (i.e., interrupt processing), the timing of the data, and the parallelism of the tasks (processes)
that handle the data
o Types
Task testing
Behavioral testing
Intertask testing
System testing
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 15 hcr:innovationcse@gg
FORMAL DESIGN REVIEWS
Participants of peer reviews
three to five participants
All the participants should be peers of the software system designer-author
Team includes
o A review leader
o The author
o Specialized professionals
A review leader
“moderator” in inspections, “coordinator’ in walkthroughs
(1) Be well versed in development of projects of the current type and familiar with its technologies.
Preliminary acquaintance with the current project is not necessary.
(2) Maintain good relationships with the author and the development team.
(3) Come from outside the project team.
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 16 hcr:innovationcse@gg
(4) Display proven experience in coordination and leadership of professional meetings.
(5) For inspections, training as a moderator is also required.
The author
The author is, invariably a participant in each type of peer review
Specialized professionals
Inspections
o A designer
o A coder or implementer
o A tester
Walkthroughs
o A standards enforcer
o A maintenance expert
o A user representative
Requisite preparations for peer reviews
Peer review leader’s preparations
To determine, together with the author, which sections of the design document are to be reviewed.
To select the team members
To schedule the peer review sessions
To distribute the document to the team members prior to the review session
Peer review team’s preparations for the review session
to read the document sections to be reviewed and list their comments before the inspection session begins
The peer review session
The presenter reads a section of the document and adds, if needed, a brief explanation of the issues
involved in his or her own words
the participants either deliver their comments to the document or address their reactions to the comments
should not deal with tentative solutions
sessions should not exceed two hours in length, or schedule for more than twice daily
Post-peer review activities
The prompt, effective correction and reworking of all errors by the designer/author and his team, as
performed by the inspection leader (or other team member) in the course of the assigned follow-up
activities.
Transmission of the inspection reports to the internal Corrective Action Board (CAB) for analysis.
o This action initiates the corrective and preventive actions that will reduce future defects and improve
productivity
Peer review efficiency
Using the following Metrics
Peer review detection efficiency (average hours worked per defect detected).
Peer review defect detection density (average number of defects detected per page of the design
document).
Internal peer review effectiveness (percentage of defects detected by peer review as a percentage of total
defects detected by the developer).
Peer review coverage
Only a small percentage of the documents and total volume of code ever undergoes peer review.
Coverage of about 5–15% of document pages still represents a significant contribution to total design quality
because the factor that determines the benefits of peer review to total quality is not the percentage of pages
covered but the choice of those pages
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 17 hcr:innovationcse@gg
INSPECTION AND WALK-THROUGH
Inspection vs. Walkthrough – participants and processes
Why it is difficult to get reviewers
Cost of poor quality is the combined cost of Prevention, Appraisal and Failure
Reviewers effort constitute “Appraisal” costs
Matter of Culture, Attitude and Expression
tremendous amount of resistance is faced in making review
most of us do not like to admit our mistakes
Holding successful reviews require us to overcome this natural resistance
In healthy SW engineering culture, reviewers understand the time spent on reviews is not wasted
lack of knowledge of review process, review techniques and cultural issues, simple resistance to change
Inspections
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 18 hcr:innovationcse@gg
Software Inspections are a disciplined engineering practice for detecting and correcting defects in software
artifacts, and preventing their leakage into field operations
An inspection is a more formal process than a walkthrough used to collect metrics or statistics about the
software process
What do inspections cover
o Inspections and walkthroughs are primarily intended to discover defects in software artifacts.
o This is a static analysis technique of software testing.
o In addition, inspections address three major tasks of process management: planning, measurement,
control.
Metrics
o Inspections are used to collect quantitative quality data at defined points in the development process.
o This can be used to give feedback to the developers, feed-forward to future development, and feed-into
future steps of process.
o Can also provide data on effectiveness of inspection techniques.
What can be inspected
o Inspections can be held a various points in development process.
o Fagan recommended inspections on:
o Detailed design
o Cleanly compiled code
o Completion of unit test
At a minimum a formal inspection includes:
o Designated moderator
o Author of the work
o At least one peer inspector
Steps of inspection
Planning
o Planning begins when entry criteria for inspection type is met.
o Moderator is selected – usually a peer or technical leader
o Selection may be made by developer, but this is generally not an ideal situation
o Management is encouraged not to look at individual inspection results
o Moderator verifies that product meets entry criteria and schedules future steps.
Overview
o Presentation to inspectors with any background information needed to properly review software
product.
o Purpose is educational only
o Data collected is author preparation time and time spent on presentation
Preparation
o Individual activity
o Author collects all material required for inspection
o Inspectors study the material and complete inspection log.
o Defects are noted at this step, but not collected
Meeting
o Meeting is conducted by moderator
o Agenda includes:
Introduction
Establishing readiness
Examining material and recording defects
Review defects
Determine disposition
Debrief
o Defect data is collected this time
o Common meeting problems
Interpersonal tensions are most likely to arise at this point
Experienced moderators can detect and defuse this tension
The more inspections that occur, the less likely interpersonal tensions are to interfere
Effort should be made by all participants to keep emphasis on producing quality product, not
making fault finding personal
Rework
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 19 hcr:innovationcse@gg
o Performed by the author in response to defect disposition determined at meeting
Follow-up
o Moderator verifies that corrections are made
o Moderator completes inspection management report and defect summary report
Inspection Roles
Author – developer of work product
Moderator – an inspector responsible for organizing and reporting on inspection
Reader – an inspector who guies the examination of the product
Recorder – an inspector who enters all the defects found on the defect list
Inspector – Member of inspection team. Often chosen to represent specific role- designer, tester, technical
writer, SQA, etc
Inspection as Process Control
When employed at various points through out the process, the completion of an inspection can trigger entry
into a new development phase.
Generally, Software Development Plan spells out entry and exit criteria and required participants in each
type of inspection.
Aspects of inspections
Initial introduction of inspection into an organization can cause anxiety and tension among developers
When it becomes clear that management supports inspection as a quality improvement technique and not a
witch hunt, the effectiveness of the inspection increases.
Inspection Data
The collection and analysis of data is what sets inspections apart from other peer review techniques such
as walkthroughs.
This data can be used in a variety of ways by a variety of personnel
Walkthroughs
Structured, team debugging, peer code reviews
a form of software peer review "in which a designer or programmer leads members of the development
team and other interested parties through a software product, and the participants ask questions and make
comments about possible errors, violation of development standards, and other problems
Walkthrough is a more informal version of an inspection
generally do not include designated moderator and are often led by the author of the software
Objective
to ensure high quality
to find
o bugs, misinterpretations, omissions, inconsistencies, ambiguities and anything that is unclear
o any thing that is complex or difficult to modify
o deviation from standards
CS621 – Software Quality Management Unit - IV
MTech CSE (PT, 2011-14) SRM, Ramapuram 20 hcr:innovationcse@gg
PEOPLE CMM
Comments & Feedback
Thanks to my family members who supported me while I spent hours and hours to prepare this.
Your feedback is welcome at [email protected]