product line engineering lecture – quality assurace...koala – component definition component is...
TRANSCRIPT
![Page 2: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/2.jpg)
© Fraunhofer IESE
1
Schedule - Lectures
![Page 3: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/3.jpg)
© Fraunhofer IESE
2
Schedule - Exercises
![Page 4: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/4.jpg)
© Fraunhofer IESE
Product Line Scoping
--- Recap ---
Component-basedPLE
![Page 5: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/5.jpg)
© Fraunhofer IESE
4
Component-based Development
Development(for reuse)
Assembly(with reuse)??
![Page 6: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/6.jpg)
© Fraunhofer IESE
5
Towards Systematic Reuse
(De-)Composition
* divide and conquer
* plug and play
* component technology asintegrationmechanism
Development for Reuse* generalization* parameterization
Development with Reuse* instantiation* adaptation
Development of generic
components
Development of specific
components
Instantiation of generic
components
Adaptation of specific
components
![Page 7: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/7.jpg)
© Fraunhofer IESE
6
Component Modeling – Generic Component Model Suite
Specification
Realization
FunctionalModel
BehavioralModel
Structural Model
Structural Model
Activity Model
Interaction Model
DecisionModel
DecisionModel
![Page 8: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/8.jpg)
© Fraunhofer IESE
7
The Principle of Locality
Run-time composition structures are traditionally described within global models
inhibits reuse
non-component oriented
The principle of locality states that
content and organization of software artifacts should be relative to a single component
Global artifacts
should be used as little as possible
should be generated recursively
![Page 9: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/9.jpg)
© Fraunhofer IESE
8
Calendar Specification: Structural
![Page 10: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/10.jpg)
© Fraunhofer IESE
9
Component Specification: Behavioral
![Page 11: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/11.jpg)
© Fraunhofer IESE
10
Component Specification: Functional
![Page 12: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/12.jpg)
© Fraunhofer IESE
11
Role of a Component Realization
To describe the way in which a component realizes the services specified in the specification
Defines a component’s
private subcomponents
architecture and design
Is defined in the context of the component’s specification
the realization models expand upon the information in the specification model
![Page 13: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/13.jpg)
© Fraunhofer IESE
12
Role of Realization Activities
Data Activities
StructuralModeling
InteractionModeling
ActivityModeling
![Page 14: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/14.jpg)
© Fraunhofer IESE
13
KOALA – component definition
Component is a unit of design, development and reuse
Interact between the environment or other components ONLY through explicit interfaces
Koala components are defined in its:IDL (interface definition language)CDL (component definition language)DDL (data definition language)
Syntactically: KOALA components are defined in ADL-like language
Semantically: KOALA components are units of computation and control connected together in an architecture
![Page 15: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/15.jpg)
© Fraunhofer IESE
14
Component Interfaces
C
m
p : I
interface I{
int Max(int x, int y);float Sin(float x);
}
component C{
provides I p;contains module m;connects p = m;within m{
p.Max(x,y) = x > y ? x : y;// p.Sin implemented in C
…}
Provides Interfaces
Has-A rather than Is-A
Ports rather than Inheritancecode-carrying
model ☺
![Page 16: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/16.jpg)
© Fraunhofer IESE
15
Connectors
C1
C2
C1
C2
C1
C2 C3
Switch Glue ModuleDirect
r
p
r
p1
r
pp2
m
![Page 17: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/17.jpg)
© Fraunhofer IESE
16
Quality Assurance (QA)Overview
![Page 18: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/18.jpg)
© Fraunhofer IESE
17
Product Line Quality AssuranceMotivation
Product lines are based on the systematic, large-scale reuseof components and other software development workproducts
Product line assets developed for reuse must have a high level of quality
because otherwise, quality problems in an asset would not just lead to a single end product with low quality, but it would propagatethis low quality into all products making use of the asset
because low-quality assets would not be reused in the long term
More than for traditional, single-system software development,quality assurance becomes a crucial part of product line engineering
![Page 19: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/19.jpg)
© Fraunhofer IESE
18
Product Line Quality AssuranceState-of-the-Art
Research in the field of product lines has mainly focused on development(analysis, design, and implementation)
Unique issues of product line quality assurance have been investigated and addressed by only few researchers so far
General techniques, methods, models, and tools for quality assurance in the context of product lines are still missing
![Page 20: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/20.jpg)
© Fraunhofer IESE
19
Product Line Quality AssuranceState-of-the-Practice
Quality assurance techniques and strategies for traditional single systemsare largely applied
The overall quality assurance effort should not be overly compromized by additional product line quality assurance
There is time pressure, and limited resources are available for quality assurance
![Page 21: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/21.jpg)
© Fraunhofer IESE
20
Product Line Quality AssuranceProblems in Practice (1/2)
Quality assurance cannot keep pace with accelerated developmentin software product line engineering
Quality assurance is already the bottleneck in single system development
Technical issues: genericity and integration
Organizational issues: more stakeholders, asynchronous communication, and distributed responsibilities
![Page 22: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/22.jpg)
© Fraunhofer IESE
21
Product Line Quality AssuranceProblems in Practice (2/2)
Quality assurance is very time and effort consuming and there are too few resources available for assuring the quality of all the products and possible combinations in a product line
Scaling of quality assurance resources often impossible
Benefits of product lines will not be achieved if quality assurance effort is increased too much
Trade-off between time-to-market and effort vs. quality
![Page 23: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/23.jpg)
© Fraunhofer IESE
22
Product Line Quality AssuranceGeneral Challenges
Make best possible use of the limited time and resources available for quality assurance
Balance the quality assurance effort between product line assetsand products
Minimize the quality assurance effort per product as well as the overall product line quality assurance effort
Select or define a quality assurance strategy most appropriate for available resources, given time schedules, and quality requirements and taking into account risks
![Page 24: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/24.jpg)
© Fraunhofer IESE
23
Quality Assurance (QA)What is quality assurance and why do we need it for product lines?
![Page 25: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/25.jpg)
© Fraunhofer IESE
24
Quality Assurance -A subprocess of Quality Management
Quality Management/Engineering/Assurance are often used as synonyms
Here, Quality Assurance is a sub-process of Quality Management
Quality management [ISO9000:2005]
Controlled activities to direct and control an organization with regard to quality
Process [IEEE829-2008]
Set of interrelated activities, which transform inputs into outputs
Quality [ISO9000:2005]
Degree to which a set of inherent characteristics fulfills requirements
Legend
(sub)activitycontrol flow
Quality ManagementQuality
PlanningQualityControl
QualityAssurance
QualityImprovement
![Page 26: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/26.jpg)
© Fraunhofer IESE
25
Quality Management –Concerns
At organizational level
Establishing a framework of organizational (development) processesand (reqirements, design & code) standards for high-quality software
At project level
Application of specific quality processes, checking that these processes have been followed, ensuring that project outputs conform to project standards
Establishing a quality plan (quality goals & standards)
![Page 27: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/27.jpg)
© Fraunhofer IESE
26
Quality Assurance –Characteristics
Quality Assurance [IEEE12207-2008]
All the planned and systematic* activities implemented within the quality system, and demonstrated as needed, to provide adequate confidence that an entity will fulfill requirements for quality
Quality assurance impactsall processes in the softwaredevelopment life-cycle,not just implementation
* systematic implies: repeatable & consistent
Quality ManagementQuality
PlanningQualityControl
QualityAssurance
QualityImprovement
Software Development
Quality Assurance
RequirementsEngineering Design Implemen-
tation
![Page 28: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/28.jpg)
© Fraunhofer IESE
27
Application Engineering (AE)
Family Engineering (FE)
Core Asset Base
RequirementsEngineering Design Implementation
RequirementsEngineering Design Implementation
Product Re-quirements
Product
Recap: Product Line Development Process
Requirements Architecture Code
![Page 29: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/29.jpg)
© Fraunhofer IESE
28
Application Engineering
Family Engineering
Core Asset Base
RequirementsEngineering Design Implementation
RequirementsEngineering Design Implementation
Product Re-quirements
Product
Product Line Development & Quality Assurance
Requirements Architecture Code
Quality Assurance
QA Assets
Quality Assurance
![Page 30: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/30.jpg)
© Fraunhofer IESE
29
QA Assets
Quality AssuranceSub-Activities / Techniques
QA sub-activities
Inspections
Measurement
Testing
Simulation
… and correspondingproduct line assets
Quality Assurance (in AE or FE)
Inspections TestingMeasure-ment
…Simulation
Inspect.Forms Metrics Tests
Simu-lations
…
![Page 31: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/31.jpg)
© Fraunhofer IESE
30
Development Qualities –Single Products (AE) vs. Product Line Assets (FE)
Single Product / Component
Functionality – does the product produce the right outputs?
Efficiency – does the product consume an appropriate amount of space & time?
Ease-of-use – how easily can a product be used in development?
Product Line / Reuse Asset
Usability – does the asset have appropriate functionality, efficiency & ease-of-use?
Generality – can the asset be reused in all required product instances?
Adaptability – how much work must be done to configure the asset?
![Page 32: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/32.jpg)
© Fraunhofer IESE
31
Product Line InspectionsHow can we review product line assets?
![Page 33: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/33.jpg)
© Fraunhofer IESE
32
Inspections (Reviews)Characteristics and Benefits
Characteristics
static analysis technique that relies on visual examination of software development workproducts to detect errors, violations of development standards, and other problems [IEEE610-1990]
used to check that certain quality aspects of a software item are fulfilled
can be performed on every workproduct that is created in the software development life-cycle (e.g. requirements, design, test cases, code, etc.)
allow identification and correction of defects earlier in the software development cycle compared to testing
Benefits
Early defect detection
Reduction of costs early in the life cycle
Facilitates early feedback
Quality Assurance
Inspections TestingMeasure-ment …
![Page 34: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/34.jpg)
© Fraunhofer IESE
33
Recap: Traditional Document & Code Inspection
Steps:
Techniques:
Ad-hoc
Checklist-based
Perspective-based
Planning
Detection
Meeting
Correction
Inspector
Organizer
Author
Author orModerator or
Inspector
Document
PlanningForm
DataSummary
DefectForm
DefectCorrection
Form
IssueReport Form
CorrectedDocument
![Page 35: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/35.jpg)
© Fraunhofer IESE
34
InspectionsProduct line-specifics
Application inspections (inspections in AE)are similar to traditional single systems inspections
because they only deal with examining single products
Differences lie in
Family inspections (inspections in FE)
Must consider product line issues => variability
Product line inspection assets
Contain variability
Do not necessarily have to deal with functionality,but may also deal with construction issues (see later)
![Page 36: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/36.jpg)
© Fraunhofer IESE
35
Product Line InspectionsTwo Extreme Approaches
Brute Force Strategy
Idea: Perform all inspection activities for all possible products in family engineering, none in application engineering
Disadvantages
large amount of inspection assets
overhead
Pure Application Strategy
Idea: Perform all inspection activities for all possible products in application engineering, none in family engineering
Disadvantages
no advantages over single systems inspections
overhead
AE RE Des. Imp.
FE RE Des. Imp.Inspections
Inspections
AE RE Des. Imp.
FE RE Des. Imp.Inspections
Inspections
![Page 37: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/37.jpg)
© Fraunhofer IESE
36
Product Line InspectionsViable Approaches
Sample Application Strategy
Idea: Use one or a few sample applications to inspect the core assets. Application inspections are still needed for each application
Pros&cons: low learning effort, but overhead
Commonality & Reuse Strategy
Idea: Inspect common parts and prepare inspection assets for variable parts in family testing. Reuse both kinds of inspection assets for inspecting individual products in application inspections
Pros&cons: low loverhead, but learning effort
Combinations
Balances individual pros&cons
![Page 38: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/38.jpg)
© Fraunhofer IESE
37
Product Line MeasurementHow can we measure product line qualities?
![Page 39: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/39.jpg)
© Fraunhofer IESE
38
Measurement –Characteristics and Benefits
Characteristics
Process by which numbers or symbols are assigned toquality attributes of software entities (components, systems, processes)in the real world [Atkinson++02]
Process output: Metrics
Measures used to indicate progress or achievement [Leon05]
Types: control (e.g. avg. time for activity) or predictor (e.g. code) metrics
Benefits
Objective comparison
to make judgements about software qualities
to access the effectiveness of processes, tools, and methods
Quality Assurance
Inspections TestingMeasure-ment …
![Page 40: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/40.jpg)
© Fraunhofer IESE
39
Project Goal
Question
MeasurementGoal
Metric Raw Data
Data Collection Procedure
Results
Project Results
Interpreted ResultsValidation
0,7
3,1415926
π42
0,7
3,1415926
π42
GQ
MGQM Plan
MeasurementPlan
MeasurementResults
Measurement –Goal-Question-Metric (GQM) Paradigm [vanSolingen++02]
![Page 41: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/41.jpg)
© Fraunhofer IESE
40
Measurement –GQM Characteristics
Most popular mechanism for goal-oriented measurement
Process steps:
1. Formulate goals
Output: Hierarchy of goals and sub-goals
2. Refine goals to questions
Questions answer to what extent the goal has been reached
Output: List of questions
3. Quantify questions by metrics
Output: metrics suite
![Page 42: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/42.jpg)
© Fraunhofer IESE
41
Product Line Measurement –An example (1/2)
Task: evaluating product line code complexity in FE (development/evolution)
1.Sustainable PL Code Evolution
2. PL Code Complexity Adaptation
3. Size Reduction
4. Shape Alignment
5.Variability Emphasis
6. VarMgtConsistency
7. Reuse Efficiency
1. Formulation of goals
Output: goal hierarchy of 7 goals
software developerfrom the viewpoint of the
variable partswith respect to
emphasizingfor the purpose of
code of software product linesAnalyze the
Q14: How many variable parts are visible at the module level? (How many should be?)Q15: How many variable parts are visible module-internally? (How many should be?)Q16: How many variable parts are indistinguishable from common code?
2. Refinement of goalsto questions
Output: 23 questionsexcerpt for goal 5:
5.Variability Emphasis
![Page 43: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/43.jpg)
© Fraunhofer IESE
42
Product Line Measurement –An example (2/2)
3. Refinement of questions to metrics
Output: metrics suite of 21 PL complexity metrics
excerpt for goal 5 / questions 14-16:
G Q Metric name Description
5 Variability emphasis
14 NVPrte Number of externally visible variable parts
15 NVPrti Number of internally visible variable parts
16 NVPrta Number of ambiguous variable parts
Other PL metrics:reuse ratio# of variation
points# of defaults
code churn among alternativesdepth of reuse hierarchywidth of reuse hierarchy
![Page 44: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/44.jpg)
© Fraunhofer IESE
43
TestingHow can we test product line assets?
![Page 45: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/45.jpg)
© Fraunhofer IESE
44
Testing –Characteristics
Characteristics [IEEE829-2008]
Process of uncovering evidence of defects in software systems
Necessary part of any quality assurance process
Defect (here: general term used for all types of quality deficits)
Non-fulfillment of a requirement, related to an intended or specified use [ISO9000:2005]
Other defect types [Jalote05]
Error: Discrepancy between a computed, observed, or measured value and the true, specified, or theoretically correct value
Failure: Inability of a system or component to perform its required functions according to its specifications
Fault: Condition that causes a system to fail in performing its required function
Quality Assurance
Inspections TestingMeasure-ment …
![Page 46: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/46.jpg)
© Fraunhofer IESE
45
Test ProcessTraditional (V-Model)
RequirementsEngineering
Requirements
Architecture
Code UnitTests
IntegrationTests
SystemTests
Test LevelsTest Items
IntegrationTesting
SystemTesting
Design
Implemen-tation
DevelopmentProcesses
TestProcesses
Legend
activitywork productproduct flowcontrol flowv&v 1 flow
1 verification & validation(Verific.: Are we building the product right? Valid.: Are we building the right product?)
Unit Testing
![Page 47: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/47.jpg)
© Fraunhofer IESE
46
TestingConcepts
Test Level Separate test effort that has its own documentation and resources [IEEE829-2008]
Test ItemSoftware or system item that is an object of testing [IEEE829-2008]
Test Case Set of test inputs, execution conditions, and expected results developed for a particular objective [IEEE610-1990]
Test Oracle Mechanism, different from the program itself, that can be used to check the correctness of the output of the program for the test cases. The output of the [program/software under testing and the test oracle] is compared to determine if the program behaved correctly for the test cases [Jalote05]
![Page 48: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/48.jpg)
© Fraunhofer IESE
47
Test Process
TestingInputs, Process, and Output
Test Inputs Test Output
TestCases
TestItems
TestOracle
Comparator
TestResults
![Page 49: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/49.jpg)
© Fraunhofer IESE
48
TestingProduct line-specific process issues
Application testing (testing in AE)is similar to single systems testing
because it only deals with the functionality of single products
Differences lie in
Family testing (testing in FE)
Must consider product line issues => variability
Product line test assets
Contain variability
Do not necessarily have to deal with functionality (execution time behaviorof complete components), but may also deal with construction issues(of incomplete system parts)
![Page 50: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/50.jpg)
© Fraunhofer IESE
49
Product Line TestingConcepts
Application Testing
Sub-process of application engineering where family test assets are reused to uncover the evidence of defects in the application
Family Testing
Sub-process of family engineering where the existence of defects in product line development assets is uncovered and where reusable test assets are created
Test Assets
Inputs and outputs of the family test process containing plans, specifications, test cases, test oracles, and test results
Are reused in application testing
![Page 51: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/51.jpg)
© Fraunhofer IESE
50
Product Line TestingTwo Extreme Approaches [Pohl++05]
Brute Force Strategy
Idea: Perform all test activities for all possible products in family engineering, none in application engineering
Disadvantages
large amount of test assets
overhead
Pure Application Strategy
Idea: Perform all test activities for all possible products in application engineering, none in domain engineering
Disadvantages
no advantages over single systems testing
overhead
AE RE Des. Imp.
FE RE Des. Imp.Testing
Testing
AE RE Des. Imp.
FE RE Des. Imp.Testing
Testing
![Page 52: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/52.jpg)
© Fraunhofer IESE
51
Product Line TestingViable Approaches
Sample Application Strategy [Pohl++05]
Idea: Use one or a few sample applications to test the core assets. Application testing is still needed for each application
Pros&cons: low learning effort, but overhead
Commonality & Reuse Strategy [Pohl++05]
Idea: Test common parts and prepare test assets for variable parts in family testing. Reuse both kinds of test assets for testing individual products in application testing
Pros&cons: low loverhead, but learning effort
Construction Testing [IESE]
Idea: Do not test the functionality of test items (“execution testing”), but the way they are constructed and compiled. A test succeeds if the test item compiles
Pros&cons: applicable in parallel with all other test approaches, low learning effort, low overhead, applicable to non-executable test items, behavior untested
[1] Pohl++05: Software Product Line Engineering
![Page 53: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/53.jpg)
© Fraunhofer IESE
52Execution Testing
Construction Testing
Product Line TestingGeneral Process
ExecutionTest Cases
ExecutionTest Oracle
ExecutionComparator
ExecutionTest Results
ConstructionTest Cases
ConstructionTest Oracle
ConstructionComparator
ConstructionTest Results
PL TestItems
![Page 54: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/54.jpg)
© Fraunhofer IESE
53
Summary
![Page 55: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/55.jpg)
© Fraunhofer IESE
54
Summary
Quality assurance is a sub-process of quality management
Quality assurance techniques are inspections, measurement, testing, etc.
Software testing is a necessary part of quality assurance, uncovering theevidence of defects in software systems
Quality assurance and testing exist for all development phases, not just for
the implementation phase
Inputs of test processes are test cases, test oracles and test items, theoutput is test results
Product line testing differs from single systems testing by testing activities
in family engineering, and test assets that may contain variability
Viable test approaches are execution tests (Sample Application Strategyxor Commonality & Reuse Strategy), and construction tests (which
succeedif the test item can be constructed/preprocessed/compiled)
![Page 56: Product Line Engineering Lecture – Quality Assurace...KOALA – component definition Component is a unit of design, development and reuse Interact between the environment or other](https://reader033.vdocument.in/reader033/viewer/2022042000/5e6d71177be65553567c13f2/html5/thumbnails/56.jpg)
© Fraunhofer IESE
55
References
[ISO9000:2005] European Standard – Quality Management Systems
[IEEE829-2008] Standard for Software and System Test Documentation
[IEEE12207-2008]Standard for Software Life Cycle Processes
[Jalote05] P. Jalote: An Integrated Approach to Software Engineering
[IEEE610-1990] Standard Glossary of Software Engineering Terminology
[Pohl++05] Software Product Line Engineering, Springer-Verlag
[vanSolingen++02] GQM Approach, Encyclopedia of SE