proficience - sqam - module 4

8/14/2019 Proficience - SQAM - Module 4

1/26

SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 11

Module 4Module 4Software MetricsSoftware Metrics


2/26


When you can measure what you areWhen you can measure what you are

speaking about and express it inspeaking about and express it innumbers, you know something about it;numbers, you know something about it;

but when you cannot measure, whenbut when you cannot measure, when

you cannot express it in numbers, youryou cannot express it in numbers, your

knowledge is of a meager andknowledge is of a meager andunsatisfactory kind; it may be theunsatisfactory kind; it may be the

beginning of knowledge, but you havebeginning of knowledge, but you have

scarcely, in your thoughts, advanced toscarcely, in your thoughts, advanced to

the stage of science Lord Kelvinthe stage of science Lord Kelvin

You cannot control what you cannotYou cannot control what you cannot

measure - DeMarcomeasure - DeMarco


3/26


MeasurementMeasurement

Economic measurements determine priceEconomic measurements determine price

and pay increasesand pay increases

Measurements in radar systems enable usMeasurements in radar systems enable us

to detect air craftto detect air craft Measurements enable doctors to diagnoseMeasurements enable doctors to diagnose

specific illnessesspecific illnesses

Measurements in atmospheric systemsMeasurements in atmospheric systemsbasis for weather predictionbasis for weather prediction


4/26



A process by which numbers orA process by which numbers or

symbols are assigned to attributes ofsymbols are assigned to attributes of

entities in the real world in such aentities in the real world in such a

way as to describe them according toway as to describe them according to

clearly defined rules.clearly defined rules.


5/26


Measurement helps..Measurement helps..

Understand about software products,Understand about software products,

processes and servicesprocesses and services

Evaluate against the establishedEvaluate against the established

standards and goalsstandards and goals

Control resources and processesControl resources and processes

Predicts the attributes of software inPredicts the attributes of software infuturefuture


6/26


Information, Data & MetricsInformation, Data & Metrics


7/26


Measurement at differentMeasurement at different

levelslevels


8/26


Use of Measurement at different levelsUse of Measurement at different levels


9/26



CharacteristicsCharacteristics Simple and robustSimple and robust

Be a part of the processBe a part of the process

RepeatableRepeatable Easy to measureEasy to measure

Non threateningNon threatening

Non evaluativeNon evaluative


10/26


Steps to design a metricSteps to design a metric

Identify metrics customersIdentify metrics customers Target goalsTarget goals Ask questionsAsk questions

Select metricsSelect metrics Standardize definitionsStandardize definitions Choose a measurement functionChoose a measurement function Establish a measurement methodEstablish a measurement method

Define a decision criteriaDefine a decision criteria Define reporting mechanismsDefine reporting mechanisms Determine additional qualifiersDetermine additional qualifiers


11/26


Goal Question MetricGoal Question Metric

Goal 1

Q1

Goal 2

Q2 Q3 Q4

M1 M2 M3 M4 M5 M6 M7 M8


12/26


GQM - ExampleGQM - Example

Goal: Reduce maintenance costs by 50% within a yearGoal: Reduce maintenance costs by 50% within a year

Questions:Questions:

- How much do we spend each month?How much do we spend each month?

- What is the fraction of cost on maintaining eachWhat is the fraction of cost on maintaining each

application?application?- How much money do we spend on corrective, adaptive andHow much money do we spend on corrective, adaptive and

preventive maintenance?preventive maintenance?

Metrics:Metrics:

- Total Budget on maintenance- Total Budget on maintenance

- Hours spent on for maintenance for each applicationHours spent on for maintenance for each application

- Hours spent on for every type of maintenance and costHours spent on for every type of maintenance and cost

involvedinvolved

One more example..One more example..


13/26


Class work..Class work..

Improve Schedule EstimationImprove Schedule Estimation

accuracy by 10% within 6 monthsaccuracy by 10% within 6 months

Reduce system testing defects forReduce system testing defects for

the next 3 development projectsthe next 3 development projects

Reduce time to close a defect byReduce time to close a defect by

40% within 3 months40% within 3 months


14/26


Metrics ExamplesMetrics Examples

Product size: Kloc, FP, Object classes, Screens,Product size: Kloc, FP, Object classes, Screens,Interface points, etc.Interface points, etc.

Effort and Schedule: Time sheetsEffort and Schedule: Time sheets

Cost: Cost of Quality, ReworkCost: Cost of Quality, Rework

Reviews: Preparation rate, Review Rate, DefectReviews: Preparation rate, Review Rate, DefectDensityDensity

Planning: Effort variance, schedule variance,Planning: Effort variance, schedule variance,ProductivityProductivity

Requirements:Requirements VolatilityRequirements:Requirements VolatilityDefects: Severity, Defect Density, Defect RemovalDefects: Severity, Defect Density, Defect Removal

Effectiveness, Defect Distribution, etcEffectiveness, Defect Distribution, etc

Maintenance: Turn around time, BacklogMaintenance: Turn around time, Backlog

Management IndexManagement Index


15/26


Software Measurement SinsSoftware Measurement Sins

Lack of management commitment Educate!Lack of management commitment Educate!

Measuring too much, too soon Start simple and balancedMeasuring too much, too soon Start simple and balanced

Measuring too little, too late Measure the aspectsMeasuring too little, too late Measure the aspects

together!together!

Measuring the wrong things Right audience!Measuring the wrong things Right audience! Imprecise metrics definitions Standardize!Imprecise metrics definitions Standardize!

Use the data for evaluating individuals Never!Use the data for evaluating individuals Never!

Use metrics to motivate, than understand Focus on resultsUse metrics to motivate, than understand Focus on results

Collecting data that is not used Inform or remove!Collecting data that is not used Inform or remove!

Lack of communication and training Handbook, website,Lack of communication and training Handbook, website,

short meetshort meet

Misinterpreting metrics data Do not over react!Misinterpreting metrics data Do not over react!


16/26


Establishing a measurement programEstablishing a measurement program

Problems to be tackled areProblems to be tackled are

lack of management commitmentlack of management commitment

imprecise or inappropriate objectivesimprecise or inappropriate objectives

unclear linkage to business and managementunclear linkage to business and managementgoalsgoals

differing management and staff expectationsdiffering management and staff expectations

assessment criteria not definedassessment criteria not defined

lack of feedbacklack of feedback

concerns about misuse of informationconcerns about misuse of information

cultural change neededcultural change needed

willingness to continue in the longer term.willingness to continue in the longer term.


17/26


Establishing a measurement program-Establishing a measurement program-

Step1Step11.1. Appoint an executive sponsor committed,Appoint an executive sponsor committed,dynamic, strong motivator, sets hard anddynamic, strong motivator, sets hard and

realistic targetsrealistic targets

2.2. Establish scope and goals ExamplesEstablish scope and goals Examples

identify where changes can improveidentify where changes can improve

productivityproductivity

assess the effectiveness of reviews andassess the effectiveness of reviews and

testingtesting

determine the effort spent on reworkdetermine the effort spent on rework

identify trends in defectsidentify trends in defects


18/26


Establishing a measurement program-Step 3Establishing a measurement program-Step 3

3. Define Key Performance Indicators, measures, models 3. Define Key Performance Indicators, measures, models

ProductivityProductivity

Function Points per person monthFunction Points per person month

QualityQuality

percentage effort spent on reworkpercentage effort spent on rework

percentage effort spent on reviewspercentage effort spent on reviews

effectiveness of reviewseffectiveness of reviews

effectiveness of systems testingeffectiveness of systems testing

average effort to correct defects found during developmentaverage effort to correct defects found during development

percentage bad fixes during developmentpercentage bad fixes during development

EstimatesEstimatesEstimated vs. actual effort (variance)Estimated vs. actual effort (variance)

Estimated vs. actual timescale (variance)Estimated vs. actual timescale (variance)


19/26



4. Establish a data collection Infrastructure4. Establish a data collection Infrastructure

who collects data?who collects data?

when is it collected?when is it collected?

where is it collected from?where is it collected from? how is it collected and validated?how is it collected and validated?

who analyses and presents the results?who analyses and presents the results?


20/26



5. Explain the Program:5. Explain the Program:

Workshop for managersWorkshop for managers

Workshop for project teams andWorkshop for project teams and

developersdevelopersTraining for staff in measurement andTraining for staff in measurement and

analysisanalysis

Questions to be answered:Questions to be answered:

Do teams understand the purpose of theDo teams understand the purpose of theprogram?program?

Do teams support the program?Do teams support the program?

Do teams know what is expected of them?Do teams know what is expected of them?


21/26



6. Devise success criteria Periodic review6. Devise success criteria Periodic review

Is the right information being generated atIs the right information being generated at

thethe

right time?right time? What is the quality of the information?What is the quality of the information?

Have baselines and targets been set for eachHave baselines and targets been set for each

KPI?KPI?

Have we got quantifiable information to aidHave we got quantifiable information to aiddecision making?decision making?

Has information been acted upon?Has information been acted upon?

Is it understood when to act on theIs it understood when to act on the

information?information?


22/26



7. Set baselines and targets for KPI7. Set baselines and targets for KPI

Identify when projects provide data and when data isIdentify when projects provide data and when data is

assessedassessed

Establish annual (or more frequent) baselines and targets,Establish annual (or more frequent) baselines and targets,

derived from quantifiable informationderived from quantifiable information Assess performance against baselines and targetsAssess performance against baselines and targets

Use data generated in first 3-6 months to set initialUse data generated in first 3-6 months to set initial

baselinesbaselines

Base initial targets on achievements of knownBase initial targets on achievements of known

improvement programsimprovement programs

Review and revise baselines and targets when measuredReview and revise baselines and targets when measured

information becomes availableinformation becomes available


23/26



8. Develop a Feedback mechanism8. Develop a Feedback mechanism

Use effective methods of presenting andUse effective methods of presenting and

reporting resultsreporting results

Dash boards Green, Yellow, RedDash boards Green, Yellow, Red


24/26


Metrics for SMEMetrics for SME

Keep it simpleKeep it simple

Customize to local needsCustomize to local needs

Be sure that it adds valueBe sure that it adds value

ScheduleSchedule EffortEffort

Defect Removal EffectivenessDefect Removal Effectiveness


25/26


Cost of Software MetricsCost of Software Metrics

For small groups, initial cost is 3-8%For small groups, initial cost is 3-8%

then drops to less than 1% afterthen drops to less than 1% after

familiarityfamiliarity

In major organizations, initial cost isIn major organizations, initial cost is

relatively high then it drops to lessrelatively high then it drops to less

than 5%.than 5%.

Source: Software Metrics, Fenton & FleegerSource: Software Metrics, Fenton & Fleeger


26/26

proficience - sqam - module 4

Documents