1/24

Impact of design decisions on software quality

Wiebe Hordijk

12-10-2004

2/24

Goal of this talk

• Show you my research framework

• Show you the status of my PhD study

• Discuss research design ideas

3/24

Agenda

• Introduction

• Overall goal

• Theoretical framework

• Related work

• Research designs

4/24

Introduction

Job

daysPerWeek = 3title = “ICT Consultant”

Project

client = “Ministry of Foreign Affairs”interesting = true

CurrentAssignment

PhD study

daysPerWeek = 2startDate = 1-1-2004plannedEndDate = 1-1-2008

mutualBenefit

5/24

Overall goal

To investigate correlations between design decisions and software quality

To build better softwareTo build softwaremore predictably

Why?

TheoreticalFramework

EmpiricalResearch

Case studyResearch

How?

Form:

6/24

Example hypotheses

• Layered systems are more reusable than non-layered systems.

• Systems with a relational database are more reliable than systems with file storage.

• In the majority of systems above 500 function points, systems with a domain model have better changeability than systems with a transaction script.

7/24

Theoretical framework

• Why?

• Reusable Rationale Blocks

• Design Space outline

• Quality Attributes & Indicators

• Examples

8/24

Theoretical framework: Why

Research

Framework

Hyp. 1 Hyp. 2 Hyp. 3

Enables Refines

Web Site

Projects

Help Feedback

Problem P

Option O2

Quality Indicator QI A x

Option O1

Option O3

x x

x

x

x

x x

x

Hypothesis H1

Hypothesis H2 Quality Indicator QI B

Quality Indicator QI C

Support

Validity Context

Problem-specific evaluation

Reusable Rationale

Block

Scientific research

+

+

+

-

-

-

Sub-problems

Actual Context

Actual Problem

instance of

instance of

10/24

Quality Attributes

11/24

Quality Indicator example

12/24

Design Space outline example

13/24

Effects Table example: Data Storage

14/24

Example: Sorting Algorithm

15/24

Framework Class Diagram

16/24

Related work

• Patterns

• SEI Reasoning Frameworks

• Other design space theories– QOC– DRL– IBIS

17/24

Research designs

• Criteria

• Difficulties

• Possible solutions

• Examples

18/24

Criteria for research designs

• Construct validity, internal validity, external validity, reliability

• Conclusion must corroborate, reject or refine hypothesis

• Research must be feasible• Hypothesis must be:

– Non-trivial– Interesting to practitioners

19/24

Difficulties in this research

• Long chains of influence

• Many rival hypotheses

• Many variables

• Little data

• Finding data costs much effort

20/24

1: Reasoning Frameworks

• Described by SEI• Deduct system properties from component

properties• Example: Failure probability with n-version

majority voting:

21/24

Drawbacks of Reasoning Frameworks

• Only available for some attributes

• Do not start from design choices, but from component properties

22/24

2: Intermediate indicators« designProblem »Domain Logic Structure• «option» Domain Model• «option» Table Module• «option» Transaction Script

« qualityIndicator »Change EffortHypothesis

« intermediateIndicator »# Lines of Code changed

« intermediateIndicator »Analysis Effort

23/24

Intermediate Indicators for Reliability

• number of test cases/source lines of code (R1) ;

• number of test cases/number of requirements• (R2);• test lines of code/source lines of code (R3);• number of asserts/source lines of code (R4).• code coverage (C)• “Reliability = 0.669 + 1.586*R1 + 0.0513*R2 –

0.0290*R3 + 0.192*R4 – 0.0774*C”N. Nagappan, 2004

24/24

3: Rationale Harvesting

• Lots of consultants and system designers document rationale

• This rationale can be generalized into rationale blocks

• The support for this rationale must be documented

• This will only yield weak support

25/24

Impact of design decisions on software quality

Wiebe Hordijk

12-10-2004

Thank you!