Open BQROpen BQRa framework for the assessment of Open Source Software

Davide Taibi - Luigi Lavazza - Sandro MorascaUniversity of Insubria

Goals

O.S.S. Comparison

Ideology

Opinion

“religion” issue

Assessment Errors

Big Projects+

• Experience: inaccurate

• Existing methods: very recent

• No methods yet consider the following:– Internal qualities (eg. code size, complexity)– External qualities (eg. fault proneness)– Support availability in the future

Current situation

Goals

COMPARISON MODEL• simple

• “formal”

Fast qualification and selection

FUNDAMENTAL ASPECTS- Functional coverage

- Number of bugs & time needed to solve- Availability of future support

- Other aspects

Preliminary analysis

O.S. Communities Contacts• GUFPI-ISMA Software Metrics • Java Open Business• International Function Point User Group (IFPUG)• CMMi Process Improvement Community

Software Metrics• Function Points• Cosmic FFP• COCOMO

Existing OSS comparison methods • Open Source Maturity Model • Open Business Readiness Ratings • Qualification and Selection of O.S. Software

Forum

Workshop

Surveys

Open BQR

OSMMOpen Source Maturity Model

Open BRROpen Business Readiness Rating

+ QSOSQualification and Selection of O.S. Software

+Open BQR

Open Business Quality Rating

OPEN SOURCE SOFTWARE COMPARISON

1. Selection

2. Parameter evaluation

3. Classification

v

vv

x

x

xx

Open BQR

1. Quick Assessment Filter

2. Data Collection & Processing

3. Data Translation

Open BQR – method

• License • Compliance with standards• Implementation language • Internationalization support • User’s manual availability • Supervised by analysts

• Specification analysis • Functional coverage assessment

• Indicator Analysis • Target Usage parameters

Open BQR – method

1.Quick Assessment Filter

• Specification analysis • Functional coverage assessment

• Indicator Analysis • Target Usage parameters

• Bug database analysis

• Closed bugs/ bug number• Bug time correction• Donations/bug number

Open BQR – method

1.Quick Assessment Filter

• Specification analysis • Functional coverage assessment

• Indicator Analysis • Target Usage parameters

• Bug database analysis • Internal qualities analysis

• Complexity (Mc Cabe)• Reuse• Dependencies

Open BQR – method

1.Quick Assessment Filter

• Specification analysis • Functional coverage assessment

• Indicator Analysis • Target Usage parameters

• Bug database analysis • Internal qualities analysis • Community activity analysis

• Number of release • Number of company supporting the product • Average number of developers per company • Independent developers

Open BQR – method

1.Quick Assessment Filter

1.Quick Assessment Filter

2.Data Collection & Processing

3.Data TranslationTarget usage parameters

External qualitiesFunctionality

Cost

Support

Open BQR – method

• Specification analysis • Rank of indicators

• Weight normalization • Indicator assessment

• Final score calculation• Result plotting Internal qualities

Test

Drupal

Mambo

WebGUI

CMS ERP

Compiere

Open BQR – CMS

Custom requirements

Personal web site:

• User-defined layout• Creation of new public or hidden page by the user• Image gallery• Donwloads • Upload of files and images via browser • Administration interface, better if in the user’s language

Open BQR – CMS

INITIAL CHOICE

• MAMBO

• DRUPAL

• WEB GUI

FUNCTIONAL COVERAGE CHECK

Customer indicator weight1. User-defined layout 10/10

2. CRUD (Create, Read, Update, Delete) pages 10/10

3. Image gallery 7/10

4. CRUD (Create, Read, Update, Delete) files and download page 5/10

5. Italian interface supported 5/10

Open BQR – CMS

Parameter Assessment - Mambo

Indicator Weight Normalized weight Score Final

Score

Target usage assessment

License 10 7,35 10 7,35%

Respect of standards 5 3,68 8 2,94%

Implementation language 8 5,88 20 11,76%

Internationalization support 5 3,68 8 2,94%Technical documentation availability 10 7,35 7 5,15%

Supervised by analysts 2 1,47 0 0,00%

Open BQR – CMS

Parameter Assessment - Mambo

Indicator Weight Normalized weight Score Final

Score

Target usage assessment

License 10 7,35 10 7,35%

Respect of standards 5 3,68 8 2,94%

Implementation language 8 5,88 20 11,76%

Internationalization support 5 3,68 8 2,94%Technical documentation availability 10 7,35 7 5,15%

Supervised by analysts 2 1,47 0 0,00%

Open BQR – CMS

Parameter Assessment - Mambo

Indicator Weight Normalized weight Score Final

Score

Target usage assessment

License 10 7,35 10 7,35%

Respect of standards 5 3,68 8 2,94%

Implementation language 8 5,88 20 11,76%

Internationalization support 5 3,68 8 2,94%Technical documentation availability 10 7,35 7 5,15%

Supervised by analysts 2 1,47 0 0,00%

Open BQR – CMS

Parameter Assessment - Mambo

Indicator Weight Normalized weight Score Final

Score

Target usage assessment

License 10 7,35 10 7,35%

Respect of standards 5 3,68 8 2,94%

Implementation language 8 5,88 20 11,76%

Internationalization support 5 3,68 8 2,94%Technical documentation availability 10 7,35 7 5,15%

Supervised by analysts 2 1,47 0 0,00%

Open BQR – CMS

Functionality

Support availabilityExternal qualities

Evaluation

Target usage indicators

Tool

Conclusions & future work

• Apply Open BQR to CMS projects

• Refine/extend the set of parameters that are relevant for the evaluation

• Reduce subjectivity as much as possible

• Disseminate the method among the O.S. users and developers

THANKS.

QUESTIONS?

Open BQR

Q & A