measuring software product quality: the iso 25000 · pdf filemeasuring software product...
Post on 17-Mar-2018
239 Views
Preview:
TRANSCRIPT
Sponsored by the U.S. Department of Defense© 2004 by Carnegie Mellon University
Pittsburgh, PA 15213-3890
Measuring Software Product Quality:the ISO 25000 Series and CMMI
European SEPGJune 14, 2004
Dave Zubrow
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 2
Objectives
Provide status on a new Software Product QualityMeasurement standard and its connection to CMMI
Provide ideas on how to get started with Software ProductQuality Measurement today
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 3
Outline
Background and Overview
Concepts and Models
Software Product Quality Measurement
Summary
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 4
Achieving Quality SoftwareRequires planning and intentional design
More than achieving the desired functionality
Must explicitly attend to both functional and non-functionalrequirements
Need to verify all requirements are being met throughoutthe life cycle
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 5
CMMI Definition for Quality Requirements
The phrase “quality and process-performanceobjectives” covers objectives and requirements forproduct quality, service quality, and processperformance. Process performance objectives includeproduct quality.
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 6
Requirements DevelopmentThis process area describes three types of requirements:• customer requirements (quality in use)• product requirements (external quality attributes)• product-component requirements (internal quality
attributes)
Taken together, these requirements address the needs ofrelevant stakeholders, including those pertinent to variousproduct life-cycle phases (e.g., acceptance testing criteria)and product attributes (e.g., safety, reliability,maintainability).
Requirements also address constraints caused by theselection of design solutions (e.g., integration ofcommercial off-the-shelf products).
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 7
Requirements Development Goals
SG 1 Develop Customer RequirementsStakeholder needs, expectations, constraints,and interfaces are collected and translated intocustomer requirements.
SG 2 Develop Product RequirementsCustomer requirements are refined andelaborated to develop product and product-component requirements.
SG 3 Analyze and Validate RequirementsThe requirements are analyzed and validated,and a definition of required functionality isdeveloped.
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 8
Process Management and PerformanceThe organization’s process needs and objectivescover aspects that include the following:• characteristics of the processes• process performance objectives, such as time to
market and product quality• process effectiveness
A quantitatively managed process is institutionalizedby doing the following:• controlling the process using statistical and other
quantitative techniques such that product quality,service quality, and process performance attributesare measurable and controlled throughout theproject (internal and external quality measures andcriteria)
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 9
Key Points in Relationship of CMMIand ISO 9126/25000 - 1
CMMI takes a total life cycle view and is inclusive in itsapproach to requirements development.
Requirements development explicitly seeks to havethe developer consider quality requirements.
Project and Process Management processes explicitlyconsider product quality as process performanceobjectives.
Neither the standard nor CMMI endorses a uni-dimensional view of quality.
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 10
Key Points in Relationship of CMMIand ISO 9126/25000 - 2Product Quality Requirements are transformed intodesigns and implemented via the Technical Solutionand Product Integration process areas.
The implementation of Product Quality Requirementsare monitored and confirmed via the ProjectManagement, Verification, and Validation processareas.
CMMI acknowledges the need for interaction andperhaps iteration among the related process areas tosatisfactorily identify, specify, and address ProductQuality Requirements.
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 11
Product QualityRequirements
Product QualityEvaluation
determines
Product QualityMeasurement
supports supports
CustomerUser
Development Organization
provide
AcquirerEvaluatorDeveloper
performguidance guidance
Relating Requirements, Evaluation,and Measurement
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 12
Outline
Background and Overview
Concepts and Models
Software Product Quality Measurement
Summary
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 13
ISO/IEC 2501nQuality Model
Division
ISO/IEC 2501nQuality Model
Division
ISO/IEC 2504nQuality
EvaluationDivision
ISO/IEC 2504nQuality
EvaluationDivision
ISO/IEC 2502nQuality Metrics
Division
ISO/IEC 2502nQuality Metrics
Division
ISO/IEC 2500nProduct Quality General Division
ISO/IEC 2500nProduct Quality General Division
Planning and Management
Planning and Management
General Overview and Guide to the SQuaRE
General Overview and Guide to the SQuaRE
SQuaRE: Architecture
ISO/IEC 2503nQuality
RequirementDivision
ISO/IEC 2503nQuality
RequirementDivision
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 14
Stakeholders’Needs in theirMinds
Stated,Implied orUnawareNeeds
Collected andIdentifiedStakeholders’(Business)Needs
Selected andSpecifiedNeeds & QIURequirements
FunctionalRequirements
ExternalQualityRequirements
FunctionalDesign &InternalQualityRequirements
NonFunctionalDesign &InternalQualityRequirements
So
licit & Id
entify
Select &
Sp
ecify
Needs and Requirements
Internal and External Quality Requirements may be stated in codingstandards, project quality goal statements, process descriptions (e.g., exitcriteria), test case descriptions, etc. They need not be explicitly identified asrequirements.
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 15
The Product Quality MeasurementReference Model
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 16
Quality In Use Model (ISO/IEC 9126)
Quality In UseQuality In Use EffectivenessEffectiveness
SafetySafety
ProductivityProductivity
SatisfactionSatisfaction
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 17
•Functionality
•Reliability
•Usability
•Efficiency
•Maintainability
•Portability
Subcharacteristics
Security
Replaceability
Testability
Resource utilization
Operability
Quality Characteristics
Suitability Accuracy Interoperability
Maturity Fault tolerance Recoverability
Understandability Learnability
Time behavior
Analyzability Changeability Stability
Adaptability Installability Co-existence
Attractiveness
Compliance
Compliance
Compliance
Compliance
Comp
Comp
Internal and External SoftwareQuality Model (ISO/IEC 9126)
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 18
Outline
Background and Overview
Concepts and Models
Software Product Quality Measurement
Summary
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 19
Quality Model Elements andMeasurement Model Elements
Attribute
Subcharacteristic
Characteristic
MeasurementPrimitive
Quality Measure
Quality Measure
Conceptual Model Operationalization
represents
represents
represents
One or moreproduce
One or moreproduce
comprise
comprise
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 20
Relating the Quality and MeasurementModelsSoftware Product
Quality
QualityCharacteristics
QualityCharacteristics
QualityCharacteristicsQuality Sub
-characteristics
QualityAttribute
QualityAttributes
Quality EvaluationReport
AssessmentAnalysis Rating
Quality Measures
Function(Formula)
MeasurementPrimitives
MeasurementMethod
Quality Attribute
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 21
Relating the Quality Measurement Model to theISO Software Measurement Process (15939)
Quality EvaluationReport
AssessmentAnalysis Rating
Quality Measures
Function(Formula)
MeasurementPrimitives
MeasurementMethod
Quality Attribute
InformationProduct
Interpretation
Indicator
Analysis Model
Derived Measures
Measurement Method
Attribute
Measurement Function
Base Measures
Analysis2503025040
Definition&
Collection2502n
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 22
CMMI Measurement & AnalysisProcess Area Goals
Align Measurement and Analysis Activities
Provide Measurement Results
Institutionalize a Managed Process
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 23
Activities for Goal 1Align Measurement and Analysis Activities• Establish Measurement Objectives• Specify Measures• Specify Data Collection and Storage Procedures• Specify Analysis Procedures
Note: The first two practices directly address the need totranslate from the conceptual to the operational.
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 24
Activities for Goal 2Provide Measurement Results• Collect Measurement Data• Analyze Measurement Data• Store Data and Results• Communicate Results
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 25
CollectData
Analyze Data
INDICATOR TEMPLATE
ObjectiveQuestionsVisual Display
Interpretation
Evolution
Assumptions
X-referenceProbing Questions
Input(s)Data Elements
Responsibilityfor Reporting
Form(s)Algorithm
80
2040
60
100
Measurement Goal #_____:
Mapping ofM&APracticesto IndicatorTemplate
INDICATOR TEMPLATE
ObjectiveQuestionsVisual Display
Interpretation
Evolution
Assumptions
X-referenceProbing Questions
Input(s)Data Elements
Responsibilityfor Reporting
Form(s)Algorithm
80
2040
60
100
Measurement Goal #_____:INDICATOR TEMPLATE
ObjectiveQuestionsVisual Display
Interpretation
Evolution
Assumptions
X-referenceProbing Questions
Input(s)Data Elements
Responsibilityfor Reporting
Form(s)Algorithm
80
204060
100
Measurement Goal #_____:
EstablishMeasurement
Objectives
SpecifyData
CollectionProcedures
SpecifyMeasures
SpecifyAnalysis
Procedures Communicate
Results
StoreData &Results
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 26
Measuring External Quality to ManageSoftware DevelopmentQuality Characteristic/Subcharacteric: Efficiency/Time BehaviorOperational Measure: Response Time
Objective: Track satisfaction of user requirement for systemresponse time.
Questions: What is the system response time with respect tocommon transaction? What is the variability in response time?
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
Test 1
Test 3
Test 5
Test 7
Test 9
Test 1
1Tes
t 13
Test 1
5Tes
t 17
Test 1
9Tes
t 21
Test 2
3Tes
t 25
Test 2
7Tes
t 29
Res
po
nse
Tim
e
Obs Avg Target
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 27
Outline
Background and Overview
Concepts and Models
Software Product Quality Measurement
Summary
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 28
Summary
Measurement links the specification of requirements toacceptance criteria
Quality is conceptual; measurement is operational.
GQ(I)M provides a means for moving from the conceptualto the operational.
The ISO 25000 series and the GQ(I)M Indicator Templatetogether can help with your implementation of CMMIRequirements Development, Verification, and Validation.
© 2004 by Carnegie Mellon University
Carnegie MellonSoftware Engineering Institute
page 29
Contact Information
Dave Zubrow3118 SEI4500 Fifth AvePittsburgh, Pa 15213USA
+1-412-268-5243 (v)+1-412-268-5758 (f)
dz@sei.cmu.edu
top related