planning for success quality management
DESCRIPTION
Understanding the who, what, why, and when of quality is essential in implementing an effective Quality Program. It requires a combination of distinct disciplines: Quality Assurance, Quality Control, and Quality Improvement. They are three unique disciplines which, when used together, can improve the efficiency and effectiveness of any organization leading to reduced cost and increased customer satisfaction.TRANSCRIPT
Software Quality ManagementJolene Eichorn
AGENDA Pursuit of Quality Quality Assurance Quality Control Quality Improvement Implementing a Quality Management Program Business Value
THE PURSUIT OF QUALITY Quality is something that is taken for granted when you
have it and cursed when you don't Quality is often not seen as a value add Quality is often neglected to meet schedule constraints Quality builds giants and nearly destroys them (Toyota) Quality does not happen by chance Quality must be premeditated and carefully planned to
be effective
What if we were to pursue quality? What would it look like? How would it be defined? How much would it cost ... or would it save?
CONCERN FOR QUALITY What if Quality provided
Competitive advantage? Increased customer satisfaction? Controlled costs?
What if poor Quality Was one of the most common reasons for schedule
overruns? Was implicated in close to half of all canceled
projects?
THE COST OF QUALITY• In 1996, there were 200 million calls for technical
support. At an average of about $23 per call, the industry spent about $4.6 billion on these calls.
• Over the past 7 years, the ratio of support to total employees in hardware and software companies has grown from 1 in 12 to 1 in 6.
• The average $3 million project costs companies using poor requirements practices an average of $5.87 million per project -- a $2.24 million premium. IAG Consulting's new Business Analysis
Benchmark , 110 projects at 100 companies surveyed
BALANCING EFFICIENCY & EFFECTIVENESS
Efficiency Effectiveness
Quality
Fit for purpose - service
Fit for Purpose - product
Cost
Time
Measured
Cycle Time
Reliability
Customer Driven – Value Defined
GOALIncrease the efficiency of
Solution Delivery Life Cycle activities, the effectiveness of
the participants, and the quality of the
deliverables.
DEFINITION OF QUALITY • Quality is the ability of your product or
service to conform to your customer's: wants needs expectations requirements
• Quality must be measurable when it occurs• Quality should be predictable when it occurs • Quality in design means realization of
purpose or fitness for use.
Align with RUP
Software Quality Improvement
Quality
Ensure Software Quality (SQA)
Software Quality Assurance
Software Quality ControlBusiness Value
QUALITY ASSURANCE“... a process for providing adequate assurance that the software products and processes in the product life cycle conform to their specific requirements and adhere to their established plans.” IEEE Standard 12207 “A planned and systematic means for assuring management that the defined standards, practices, procedures, and methods of the process are applied.” CMMI, SEI
“The planned and systematic activities implemented in a quality system so that quality requirements for a product or service will be fulfilled.” ASQ
The focus is on the process used to create the deliverable
QUALITY ASSURANCEIn
pu
ts
COBIT, CMMi, BABOK, PMBOK and other best practice frameworks
SDLC Methodologies
Current processes, standards, policies
Serv
ices
Developing and supporting processes
Define measures
Process management
Define standards, policies, procedures
Prevention activities
Assess quality
Deliver training
Support automation tools
Prevent defects
Outp
uts
Methods and Processes
Assessment and Metrics
Published standards, policies, procedures
Templates
Quality Assessment
Software Quality Plan
Training class
Deploy Automation tools
QUALITY CONTROL“The operational techniques and activities that are used to fulfill requirements for quality.” ISO 8402-1994
“The observation techniques and activities used to fulfill requirements for quality.” ASQ
The focus is on the deliverable itself.
QUALITY CONTROLIn
pu
ts
Requirements
Deployed software
Design documentation
Serv
ices Define test strategy
Create test plans
Configure test environment
Develop test cases
Execute unit, integration, system, performance, user acceptance tests
Conduct inspections
Outp
uts
Test strategy
Test plans
Test environment
Test cases
Test execution results
Defects
Mechanisms:Testing process, test tools
QUALITY IMPROVEMENTChanging a process to improve the reliability of an outcome. Methods for quality improvement include:
Kaizen Six Sigma IEEE TQM TRIZ CMMI ITIL
QUALITY IMPROVEMENTIn
pu
ts
Methods and processes
Assessment and metrics data
Published standards, policies, procedures
Templates
Quality assessment
Defect data
Serv
ices Improve product
Reduce variation
Reduce defects
Improve processes
Re-engineer process Outp
uts
Quantifiably improved processes, standards, templates
Mechanisms:COBIT, CMMi, 6, LeanVAL IT, ITIL, TRIZ
QUALITY IMPROVEMENTLeverage COBIT•As a way of aligning IT with the business•As a tool for defining IT process improvements•For control assessments•For maturity assessments•As a basis for performance measurement•As a communication tool•To map IT goals to business goals
Leverage VAL IT•For implementing IT-enabled change and innovation •To optimize returns from tool investments
Leverage CMMi•As a way to guide process improvement.•For helping to integrate traditionally separate organizational
functions.•To set process improvement goals and priorities.•To provide guidance for quality processes.•As a means for providing a point of reference for appraising
current processes. •To explicitly link your organization's activities to your business
objectives.
BALANCING SPEED & QUALITY
Quality
Fit for purpose - service
Fit for Purpose - product
Cost
Time
Measured
Cycle Time
Reliability
Customer Driven – Value Defined
LeanCreate ValueEliminate WasteStrategic Focus
6Improve QualityReduce VariationImprove Predictability
BALANCING SPEED & QUALITYImplementing Lean programs to reduce time and cost may be ineffective without predictable processes built through Quality Management, thereby preventing Lean goals from being met.
In addition, quality process automation can have an enormous impact on efficiency and quality.
QUALITY IMPROVEMENTLEAN PRINCIPLE 5 - PERFECTION
Pursue Continuous Improvement / Strive for Excellence of SE Processes
Do not ignore the basics of Quality: Build in robust quality at each step of the process, and resolve and do not pass along problems. Strive for perfection in each process step without introducing waste Do not rely on final inspection; error proof wherever possible If final inspection is required by contract, perfect upstream processes pursuing 100% inspection pass rate Move final inspectors upstream to take the role of quality mentors Apply basic PDCA method (plan, do, check, act) to problem solving Adopt and promote a culture of stopping and permanently fixing a problem as soon as it becomes apparent.
LEfSE were developed by the Lean Systems Engineering Working Group of INCOSE (International Council for Systems Engineering). Version 1.0 was released at the INCOSE IW, San Francisco, March 1, 2009.
QA VERSUS QC
Quality Assurance
• Defect Prevention• Proactive Process• Customer
Confidence• Preventative
Quality Control
• Defect Detection• Reactive• Producer
Confidence• Corrective
QA VERSUS QCQuality Assurance
Examples• Process Definition• Quality Assessment• Inspection (Mays, 1990)• Tool Selection• Training• Requirements Analysis• Usability Analysis• Early Prototyping• Root Cause Analysis• Process Checklists• Defect Prevention Process
Quality Control
Examples• Walkthrough• Testing• Inspection (Fagan, 1976)• Checkpoints
A QUALITY MANAGEMENT PROGRAM
Ensures that: An appropriate methodology is in place Projects use standards and procedures Independent reviews and audits are performed Documentation is produced during development Change controls are in place An inspection process is leveraged Feedback is provided to team Noncompliance issues are addressed
QUALITY ASSURANCEThe purpose of Quality Assurance is to provide staff and management with objective insight into processes and associated work products.
A QA group that is independent of the project provides objectivity Everyone performing QA activities should be trained in QA Those performing QA activities for a work product should be separate
from those directly involved in developing or maintaining the work product An independent reporting channel to the appropriate level of
organizational management must be made available QA should begin in the early phases of a project to establish plans,
processes, standards, and procedures that will add value to the project and satisfy organizational policies
CMMI, Guidelines for Process Integration and Product Improvement, Mary Beth Chrissis, Mike Konrad, Sandy Shrum
QUALITY ASSURANCE SERVICES
• Customize Project Dashboard• Develop Development Case• Develop Quality Plan• Acquire Tools• Implement and Configure Tools• Launch Development Processes• Prepare Guidelines• Prepare Templates• Process and Tools Support• Tailor Development Process
Development Environment
• Conduct Quality Assessments• Perform Retrospectives• Stage Gate Assessments• Define and Publish Quality Measures
(Dashboard)• Develop Standard Dashboard Reporting• Facilitate Inspections• Facilitate Standards Adherence
Quality
• Configure Test Environment• Maintain Test Data
Test Environment
• Coordinate Software Engineering Process Group
• Develop and Publish Guidelines• Develop and Publish Methodology and
Process• Develop and Publish Standards• Publish Templates
Methodology
• Formal Review Moderator Training• Formal Review Participant Training• Informal Review Training• Process Training• Quality Training• Tools Training• Methodology Training• UML Training
Training
• Process Mentoring• Standards Mentoring• Tools Mentoring• UML Mentoring
Mentoring
QUALITY ASSURANCE ROLES
• Responsible for all process related aspects of the organization, such as:
• Reviewing best practices. Continuously improving processes using quality measures and tools.
• Developing standard methods, processes, guidelines.
• Creating templates. • Training and mentoring.
Methodologist
• Responsible for all process related aspects of the project, such as:
• Assures that the standards, processes, and procedures are appropriate for the project and are correctly implemented.
• Tailoring the process to match the specific needs of the project.
• Educating and mentoring project members on process related issues.
• Ensuring that valuable project experience is harvested and fed back into the process.
• Assisting the Project Manager in planning the project.
Process Engineer
• Supporting the tools used by the project. This includes selecting and acquiring tools, configuring and setting them up, and verifying that they work.
• Understanding the underlying processes used by the project.
Tools Specialist
• Participating in reviews, walk-through, process checks.
• Facilitating or participating in formal inspections, ensuring the process is followed correctly.
• May sample actual requirements.• Defining quality measures. • Ensuring project quality. • Conducting quality assessments and audits
to ensure that documented standards, processes and procedures are followed.
• Establish software metrics in order to measure the effectiveness of the process.
• Note: The CMMi repeatedly states that senior management must "protect individuals performing the SQA role,” given the possibility of management retaliation against an employee who finds non-compliance issues.
Software Quality Analyst
QUALITY CONTROLThe purpose of Quality Control is to validate and verify that software program/application/products:
1. meet the business and technical requirements that guided its
design and development2. works as expected3. can be implemented with the same characteristics
A QC group that is independent of the those that developed, or maintain the work product, provide objectivity.
QC provides an objective, independent view of the software to allow the business to appreciate and understand the risks at implementation of the software.
QA and QC may address the same work product, but from different perspectives.
QUALITY CONTROL SERVICES
• Define test strategy• Estimate test effort• Manage test schedule• Define test plans
Test Planning
• Define functional test cases• Define performance test
scenarios
Test Design
• Create test cases• Execute test cases
Manual Testing
• Create automated tests• Execute automated tests
Automated Testing
• Create load and stress tests• Execute load and stress tests• Tune applications for
performance
Performance Testing
QUALITY CONTROL ROLES
• Conducts manual and automated testing, logs the outcomes and reports defects.
• Takes an active role in the verification process (all the documents or requirements), this includes the requirements inspection process and verifying that documented requirement standards are followed.
Tester
• Identifies and defines the required tests
• Monitors detailed testing progress and results in each test cycle
• Represents stakeholders who do not have direct or regular representation on the project.
Tester Analyst
• Leads in defining the test approach and ensuring its successful implementation
• Identifies the appropriate techniques, tools and guidelines to implement the required tests
• Provides guidance to the test effort
Test Designer
• Leads the overall test effort• Advocate for quality and testing• Provides resource planning and
management, and resolution of issues that impede the test effort
Test Manager
BUSINESS VALUEQuality is a differentiatorOrganizations typically build the software and systems that make them unique, while they purchase applications for automating common business functions that do not demand differentiation from competitors.
The bottom line is:
Consistent delivery of high-quality software will set an organization apart from its competition.
The business value of software quality, Geoffrey Bessin, http://www.ibm.com/developerworks/rational/library/4995.html
BUSINESS VALUEQuality enables innovationIt is quality in development and maintenance of software that makes it possible for the business to react, adapt, and deploy quickly. If there is any breakdown, any inefficiency, any lack of quality, then the organization will fail to reach the market in a timely fashion and the organization will find itself at a competitive disadvantage.
The bottom line is:
High quality software facilitates innovation by increasing predictability, lowering risk and reducing rework, resulting in happier customers and more revenue.
The business value of software quality, Geoffrey Bessin, http://www.ibm.com/developerworks/rational/library/4995.html
AND BEST OF ALL - QUALITY IS FREE!
*Philip Crosby was so convinced of this that he titled one of his books Quality is Free, since the major costs of quality are the costs associated with poor quality.
cost of defect prevention and early reviews
cost of late stage testing and
reworkCost of Quality
Increasing defect
prevention
Reduces rework
When the amount saved by avoiding defects rises faster than the amount invested to avoid defects, quality is free*
BUSINESS VALUE
Troubled ProjectPoorly defined process / inconsistent buy-in
Front-loaded requirements gathering without reassessment or validation
Absent or limited architectural and code quality assurance
Late cycle system testing
Limited post-deployment monitoring and assessment
Excess cost is easy to spot at every stage.
Perfect Project A well-defined process with unanimous buy-in from team members
Focused attention to value-add, detail and quality across the lifecycle
Constant evaluation of end-user feedback to ensure reinvigoration of the product line and satisfied, repeat customers
There is no excess cost -- the process is optimized.
Quality is freeWhat is the cost of high quality? Is high quality free?
The business value of software quality, Geoffrey Bessin, http://www.ibm.com/developerworks/rational/library/4995.html
Improved quality enables teams to deliver more projects on time, at lower cost, with more features. By preventing defects in the system throughout the entire development process, a team eliminates the time and cost required to find and fix those defects later on.
BUSINESS VALUEReworking a software requirements problem once the software is in operation typically costs 50 to 200 times what it would cost to rework the problem in the requirements stage.
Because potentially a:
1 sentence requirement
5 pages of design diagrams
500 lines of code
15 pages of user documentation
48 test cases
Boehm and Papaccio 1988
Currently, over half of all errors are not found until “downstream” or during post sale software use. This occurs even though 80% of development costs are spent on defect removal.
SAVINGS CAN BE HUGE Each hour spent on Inspections avoided an average of 33 hours of maintenance, and inspections were up to 20 times more efficient than testing. - Russell 1991, study done on large programs
Software Quality Inspections have been found to produce net schedule savings of 10 to 30%. - Gilb and Graham 1993
Every major defect found at Inspection will save 9 hours of downstream correction effort. – Reeve, 1991
Most of the 78 organizations that participated in the National Quality Experiment have seen a ROI of 2:1 to 8:1 on investment in Inspections. – O’Neill, 2001
CONCLUSIONUnderstanding the who, what, why, and when of quality is essential in implementing an effective Quality Program. It requires a combination of distinct disciplines:
They are three unique disciplines which, when used together, can improve the efficiency and effectiveness of any organization leading to reduced cost and increased customer satisfaction.
•A proactive defect prevention program
•Assures that the standards, processes, and procedures are appropriate for the project and correctly implemented.
Quality Assurance
•A reactive defect identification program
•Checks that the project follows its standards, processes, and procedures, and that the project produces the required internal and external (deliverable) products.
Quality Control
•A program that uses the result of Quality Control to improve Quality Assurance
Quality Improvement
ABOUT THE AUTHORSolution Delivery Over 20 years of experience in delivering solutions that optimize tool automation
investments, improve productivity, quality, transparency and alignment between business and IT.
Program Turnaround Specializes in requirements quality assessments and disentanglement of challenged
programs.
Capability Maturity Brings practical knowledge of requirements management best practices and promotes
requirements maturity across large organizations.
Quality Works with organizations and project teams to improve capabilities by implementing
Lean Six Sigma process improvements, effective requirements practices, dashboard reports, and formalizing requirements change management.
Contact Information:Jolene Eichorn, Software Quality Practitioner http://www.linkedin.com/in/eichorn
改善CHANGE FOR THE
BETTER