criteria for the evaluation of implemented...

Arent Janszoon Ernststraat 595-HNL-1082 LD [email protected]

Criteria for the Evaluation of Implemented ArchitecturesEric Bouwers, Joost Visser, Arie van Deursen

2009/09/22

Criteria for the Evaluation of Implemented Architectures

Why evaluate software architectures?

‘Architectures allow or preclude nearly all of thesystem’s quality attributes’

-- from `Evaluating Software Architectures` by P. Clements, R. Kazman and M. Klein


Why evaluate implemented architectures?


Why do we want criteria for the evaluation?

Maintenance Phase

Architecture Evaluation

Result


Result


Result


Result


Literature reviews

A framework for classifying andcomparing software architecture

evaluation methods.

M. Babar, L. Zhu, and D. R. Jeffery

A survey on softwarearchitecture analysis

methods.

L. Dobrica and E. Niemelä.


IEEE Computer, July 2009

‘Technology transfer remainsa major challenge. There is

limited out of the box processand tool support for

organizations that want tostart reviews.’

‘Software Architecture Review: The state of Practice’

by M. Babar, I. Gorton


Software Risk Assessment (SRA)

A one-time investigation into software maintainabilityand related business risks


Architectural properties

GUI

Data Access

Business Logic

High Level Design:

Saving Paying Stocks

Modularization:

if(hasPayed() && !isAdmin()){ showThankYou();}

Separation of Concerns:

Business logic/Security


Research questions

Q1: Which system attributes do experts take into accountwhen evaluating architectural system properties?

Q2: How do these system attributes influence thearchitectural system properties?


Empirical study resources


A selection of our clients

Financials ITLogistics OtherPublic


Evaluated systems characteristics


Empirical study design overview

CreateAttribute List

Create Mapping to Properties

ValidateAttributes &Mapping


Experimental design (1/3)Creation of the attribute list

1) Check table with property ratings for argumentsif this table does not exists

use specific paragraphs

For each report:

2) For all arguments founddetermine system attributeif the attribute is not on the list

add system attribute to the list


Experimental design (2/3)Create mapping from attributes to properties

2) For all arguments founddetermine system attributeadd one to count of attribute for property

1) Extract arguments per property

For each report:


Experimental design (3/3)Validation of Attributes and Mapping


Results of Empirical Study (1/3)





15 System attributes

• Abstraction• Layering• Logic in Database• Module Inconsistency• Module Size• Source Grouping• Technology Combination• Textual Duplication

• Functional Duplication• Libraries / Frameworks Usage• Module Dependencies• Module Functionality • Relation Documentation and Implementation• Technology Age• Technology Usage


More details available in the paper

Inspecting the call-graph on modulelevel, matching thisagainst expecteddependencies

The staticdependencies (i.e.calls, includes)between modules

ModuleDependencies

Inspecting the listof imports andstructure of thesource- and build-files

The usage ofstandard librariesand frameworks

Libraries /Frameworks

AssessmentApproach

DescriptionName


Results of empirical study (2/3)





High Level Design

Modularization

Separation of Concerns

..... .....

Module Dependencies

Module Functionality

Relation Doc. / Impl.

Functional Duplication

Technology Combination

Technology Age

7 4 2 2 5 13

11 32 3 6 1 0

6 13 0 18 0 0

Attributes used for each property


High Level Design

Modularization


..... .....

Module Dependencies





Technology Age

X X

X X X

X

X

Attributes used for each property


Results of empirical study (3/3)





Interview results

High Level Design

Modularization


E1

E1, E2

E1

E2

E2

..... .....

Module Dependencies





Technology Age


Threats to validity

Representative data

Reliability of measurements

Generalization of the results


‘What is in it for me?’-- you


Our contributions

•A description of an empirical study using over 40 reports

•The identification of 15 system attributes that have an impact on the maintainability of an architecture

•An analysis of the projection of the found system attributes onto three architectural system properties


Application of the results


Future research opportunities


Questions or comments?


• A description of an empirical study using over 40 SRAreports• The identification of 15 system attributes that have an

impact on the maintainability of an implementedarchitecture• An analysis of the projection of the found system

attributes onto three architectural system properties

Feel free to contact me:[email protected]

criteria for the evaluation of implemented...

Documents