metrics an overview. what are metrics? “ a quantitative measure of the degree to which a system,...

MetricsAn overview

What are metrics?

“ A quantitative measure of the degree to which a system, component, or process possesses a given attribute.” [IEE93]

• Attributes of metrics - Simple, persuasive, consistent, Independent and effective

What is measured

• Processes are collection of software related activities

• Products are any artifacts, deliverables or documents that result from a process activity.

Types of Metrics• Metrics for Analysis

• System Size• Functionality Delivered• Specification Quality

• Metrics for the Design• Architectural Metrics• Component Level• Interface Design

• Metrics for the Code• Halstead Metrics• Complexity Metrics• Length Metrics

• Metrics for Testing• Defect metrics

Analysis Model Metrics

• Function – Based Metrics– Function point metric

• Variables• Formula • Questions

• Metrics for Specification Quality– Specificity– Completeness

Design Model Metrics

• Architectural Design Metrics

• Object – Oriented Design Metrics

• Component – Level Design Metrics

• User Interface Design Metrics

Architectural Metrics

• Focus on characteristics of the architecture with emphasis on the structure and effectiveness of modules or components within the architecture.– Structural Complexity

• S(i) = [fout(i)] ^2– Data complexity

• D(i) = v(i) / [fout(i) + 1]– System complexity

• C(i) = S(i) + D(i)

where fout is defined as the number of modules directlyinvoked by module i. v(i) = number of input and outputvariables in or out of i.

• Design Structure Quality Index

Object Metrics• Size:

– Population, Volume, Length, Functionality

• Complexity• Completeness• Cohesion• Coupling - an indication of “connectedness” of

a module with others, global data and environment

• Volatility

9

Object-Oriented

• System size – Number of function calls and objects.

• Class or method size– Size of classes and methods

• Coupling and inheritance– Interdependence of classes. Number and

complexity of relationships.

• Class or method internals– Complexity and level of documentation

User Interface Metrics

• Layout• Absolute and relative position of entities• Frequency of use• “Cost of transition”

• Cohesion • Relative connection between on – screen to other on –

screen content

• Time • Time to achieve an specific action• Time to recover from error• Time to achieve a use case

Source Code Metrics• Lines of code metric

– Lines of code used to develop a program– Set limits on coding lines

• McCabe’s cyclomatic metrics– Path control – Logical decision statements and operators

• Halstead Metrics– Depend on the following measures:

• Number of direct operators• Number of direct operands• Number of operator occurrences• Number of operand occurrences

– Predicts • Length• Volume• Time, effort• Number of errors

Testing Metrics• Information for testing metrics can be gathered thru

various sources:– Function – based metrics– Cyclomatic complexity

• Halstead Metrics for testing

• Metrics for OO Testing– Metrics used in design provide an indication of design

quality– The metrics consider aspects of encapsulation and inheritance

13

Web Projects

Since web projects can are more dynamic than most other types of projects it can be harder to measure them.

• Number of static/dynamic pages

• Number of internal page links

• Number of executable functions (scripts, applets, etc)

Product Metrics Future

• Application Domain Complexity Research

• Automated Product Metric Tools– ESQUT (Evaluation of Software Quality from

User’s viewpoinT)– WEBMETRICS

• Mathematical Product Metrics

15

What are process metrics?

Process metrics really encompasses both process and project metrics.– Process metrics are used to measure the

effectiveness of a development process over multiple projects

– Project metrics are used to evaluate accomplishment of a single development effort

16

Process Metrics

Process Metrics are generally used at the project management level.– Strategic in nature– Main goal is process improvement– Metrics from individuals combined to

provide group/project metrics– Care must be taken to keep individual

metrics private

17

Measurement Methods

Due to the variation in software, and the lack of standards, there are many types of metrics that can be used

• Size-Oriented

• Function-Oriented

• Object-Oriented

• Use-Case Oriented

18

Size-Oriented

• Metrics are normalized based on the size of the software

• Focus on LOC (Lines of Code)– Errors Per KLOC (thousand lines of code)– Defects per KLOC– $ per LOC– Page of documentation per KLOC

19

Function-Oriented

• Metrics are normalized based on program functionality

• Most commonly used metric is the FP (function point)– Errors per Function Point– Defects per FP– $ per FP– Pages of documentation per FP– FP per person-month

20

Object-Oriented

• number of classes reused

• number of classes with defects

21

Metrics Tools

• SynQuest• NEXTRA• AMI• SPQR/20• SOFT-ORG• SQUID M-Base• Analyst4j• Eclipse Metrics

22

Why use Metrics?

"If you can not measure it, you can not improve it.“ – Lord Kelvin

• Characterize - Increase Understanding

• Evaluate

• Predict - Managing Software

• Improve - Guiding Improvements

23

Why use Metrics - Benefits

Although it can be hard to generate metrics from software, there are many benefits

• Manage complexity

• Process improvement

• Quality tracking

• Cost estimation

24

Why use Metrics - Obstacles

"Not everything that can be counted counts, and not everything that counts can be counted.“ - Albert Einstein 

• Takes time and effort away from the project

• Simple project may not benefit

• Projects may vary substantially

• Indicators must be interpreted

25

Why use Metrics – Bottom Line

Provides the best option to standardize the development process to ensure quality and reliability

26

Establishing a Metrics Program

Software Productivity Center, Inc. suggests an 8 step approach:

1. Document the Software Development Process 2. State the Goals 3. Define Metrics Required to Reach Goals 4. Identify Data to Collect 5. Define Data Collection Procedures 6. Assemble a Metrics Toolset 7. Create a Metrics Database 8. Define the Feedback Mechanism

27

Establishing a Metrics Program

Establishing a program takes planning and time to be effective.

• Need historical data from past projects to establish indicators

• Goals have to be well defined to be meaningful

• Those collecting measures need to be vested in the program

CONCLUSIONS

• Problems in products in each phase can be detected and removed at an early stage by using metrics.

• Metrics provide a quantitative and predictive view of potential problems. Thus, they are a powerful tool for product development.

• Metrics are still in a young stage. Research is needed to set a ground for prediction of problems.