cmmi conference denver, colorado 2004€¦ · evolving a measurement program for systems and...
TRANSCRIPT
-
Evolving a MeasurementEvolving a MeasurementProgram for Systems andProgram for Systems and
Software Engineering ProcessSoftware Engineering ProcessImprovementImprovement
CMMI ConferenceCMMI ConferenceDenver, ColoradoDenver, Colorado
20042004
-
Techniques for Meas Pgm - 2version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
WelcomeWelcome
Wilkommen
Bienvenido
WelKom
Bienvenue
BienvenutoVälkom
men
Tervetuloa Witamy
Huan Yín
ЌАΛΟΣΟΡΙΣΑΤ
Ε
����
-
Techniques for Meas Pgm - 3version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
MeasurementMeasurementIs It ReallyIs It ReallyNecessary?Necessary?
-
Techniques for Meas Pgm - 4version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
At Project Start, DoAt Project Start, DoYou Know…?You Know…?
�Can it be done?�How long will it take?�How much will it cost?�How many people will it take?�What is the risk?�What are the tradeoffs?�How many errors will there be?
-
Techniques for Meas Pgm - 5version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
What Do You KnowWhat Do You KnowNow?Now?
�How much does your current developmentprocess cost?
�How much value does each piece of theprocess add?
�What would the impact be of deleting,modifying, adding a procedure to the process?
�What activities contribute the most to the finalproduct cost?
�Have you tried to improve the currentdevelopment process?�What changes in cost/value resulted from that
improvement effort?
-
Techniques for Meas Pgm - 6version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
What Do You KnowWhat Do You KnowNow?Now? -- 22
�What processes represent the greatestpotential for return on improvementinvestment?
�How would you quantify the value of theprocess improvement investment?
�Do you really want to know where the moneyis going in your software developmentprojects?
�What value do you think you are delivering toyour customers? Do they agree?
�How much is the knowledge of your costs andthe value delivered worth to you?
-
Techniques for Meas Pgm - 7version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
MeasurementMeasurementand Analysisand Analysis
-
Techniques for Meas Pgm - 8version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Measurement andMeasurement andAnalysis OverviewAnalysis Overview
�A measurement initiative involves thefollowing:�Specifying the objectives of measurement and
analysis such that they are aligned with establishedinformation needs and business objectives
�Defining the measures to be used, the datacollection process, the storage mechanisms, theanalysis processes, the reporting processes, andthe feedback processes
�Implementing the collection, storage, analysis, andpresentation of the data
�Providing objective results that can be used inmaking business judgments and taking appropriatecorrective actions
-
Techniques for Meas Pgm - 9version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
EstablishEstablishMeasurementMeasurementObjectivesObjectives
�Measurement objectives�Document the purposes for which measurement
and analysis is done�Specify the kinds of actions that may be taken
based on the results of the data analyses�Continually ask the question – what value will this
measurement be to those people who will be askedto supply the raw measurement data and who willreceive the analyzed results
�Involve the end users of the measurement andanalysis results in setting measurement objectiveswhenever possible
�Maintain traceability of the proposed measurementobjectives to the information needs and businessobjectives
-
Techniques for Meas Pgm - 10version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Specify MeasuresSpecify Measures
�Identify candidate measures based ondocumented objectives and refine intoprecisely defined, quantifiable measures
�Define how data can and will be derived fromother measures
�Examples of Commonly Used DerivedMeasures�Defect density�Peer review coverage�Reliability measures
-
Techniques for Meas Pgm - 11version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Specify Data CollectionSpecify Data Collectionand Storage Proceduresand Storage Procedures
�Specify how to collect and store the data foreach required measure�Make explicit specifications of how, where, and
when the data will be collected
�Develop procedures for ensuring that the datacollected is valid data
�Ensure that the data is stored such that it is easilyaccessed, retrieved, and restored as needed
-
Techniques for Meas Pgm - 12version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Specify AnalysisSpecify AnalysisProceduresProcedures
�Define the analysis procedures in advance�Ensure that the results that will be fed back
are understandable and easily interpretable�Collecting data for the sake of showing an
assessor the data is worthless�Showing how it can be used to manage and control
the project is what counts
-
Techniques for Meas Pgm - 13version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
CollectCollectMeasurement DataMeasurement Data
�Collect the measurement data as defined, atthe points in the process that were agreed to,according to the time scale established
�Generate data for derived measures
�Perform integrity checks as close to thesource of the data as possible
-
Techniques for Meas Pgm - 14version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Analyze theAnalyze theMeasurement DataMeasurement Data
�Conduct the initial analyses�Interpret the results and make preliminary
conclusions from explicitly stated criteria�Conduct additional measurement and
analyses passes as necessary to gainconfidence in the results
�Review the initial results with all stakeholders�Prevents misunderstandings and rework
�Improve measurement definitions, datacollection procedures, analyses techniques asneeded to ensure meaningful results thatsupport business objectives
-
Techniques for Meas Pgm - 15version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Store theStore theMeasurement DataMeasurement Dataand Analyses Resultsand Analyses Results
�The stored information should contain orreference the information needed to:�Understand the measures�Assess them for reasonableness and applicability
�The stored information should also:�Enable the timely and cost effective future use of
the historical data and results�Provide sufficient context for interpretation of the
data, measurement criteria, and analyses results
-
Techniques for Meas Pgm - 16version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Communicate theCommunicate theMeasurement ResultsMeasurement Results
�Keep the relevant stakeholders up-to-dateabout measurement results on a timely basis
�Follow up with those who need to know theresults�Increases the likelihood that the reports will be
used�Assist the relevant stakeholders in
understanding and interpreting themeasurement results
-
Techniques for Meas Pgm - 17version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Measurement andMeasurement andAnalysis GroupAnalysis Group
�Consider creating a measurement group thatis responsible for supporting the Measurementand Analysis activities of multiple projects
-
Techniques for Meas Pgm - 18version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Measurement andMeasurement andAnalysis ToolsAnalysis Tools
�Incorporate tools used in performingMeasurement and Analysis activities such as:�Statistical packages�Database packages�Spreadsheet programs�Graphical packages�Packages that support data collection over
networks and the internet
-
Techniques for Meas Pgm - 19version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Measurement andMeasurement andAnalysis TrainingAnalysis Training
�Provide training to all people who will performor support the Measurement and Analysisprocess�Data collection, analyses, and reporting processes�Measurement tools�Goal-Question-Metric Paradigm�How to establish measures
� how to determine efficiency and effectiveness�Quality factors measures (e.g., maintainability,
expandability)�Basic and advanced statistical techniques
-
Techniques for Meas Pgm - 20version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Basic MeasuresBasic Measures
-
Techniques for Meas Pgm - 21version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Basic MeasuresBasic Measures
�Estimate Size and/or Complexity - a relativelevel of difficulty or complexity should beassigned for each size attribute
�Examples of attributes to estimate for SystemsEngineering include:�Number of logic gates�Number of interfaces
�Examples of size measurements for SoftwareEngineering include:�Function Points�Lines of Code�Number of requirements
-
Techniques for Meas Pgm - 22version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Basic MeasuresBasic Measures -- 22
�Determine effort and cost – use historicaldata or models
�Establish the project’s schedule based on thesize and complexity estimations
�Include, or at least consider, infrastructureneeds such as critical computer resources
�Identify risks associated with the cost,resources, schedule, and technical aspects ofthe project
�Control data (various forms of documentation)required to support a project in all of its areas.
-
Techniques for Meas Pgm - 23version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Basic MeasuresBasic Measures –– 33
�Identify the knowledge and skills needed toperform the project according to the estimates
�Select and implement methods for providingthe necessary knowledge and skills�Training (Internal and External)�Mentoring�Coaching�On-the-job application of learned skills
�Monitor staffing needs – based on effortrequired and the necessary knowledge andskills to achieve the defined tasks
-
Techniques for Meas Pgm - 24version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Basic MeasuresBasic Measures -- 44
�Involve relevant stakeholders throughout theproduct lifecycle
�Track technical performance (Completionof activities and milestones against theschedule Example:�Product components designed, constructed, unit
tested and integrated�Compare actual milestones completed vs.
established commitments�Monitor commitments and critical
dependencies against those documented inthe project plan
-
Techniques for Meas Pgm - 25version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Basic MeasuresBasic Measures -- 55
�Track quality – Problems/Defects(open/closed by product/activity)�Problems and defects are direct contributors to
the amount of rework that must be performed—asignificant cost factor in development andmaintenance
-
Techniques for Meas Pgm - 26version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Effectiveness ofEffectiveness ofProcessesProcesses
-
Techniques for Meas Pgm - 27version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Effectiveness ofEffectiveness ofProcessesProcesses
�In addition to defining the processes that wewish to follow on our project, we need toensure we are following them and we shouldbe able to determine if the processes areworking for us the way we expected them to�How well are the processes working?
-
Techniques for Meas Pgm - 28version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
RequirementsRequirementsManagementManagement ––Effectiveness ExampleEffectiveness Example
�Number of change requests per monthcompared with the original number ofrequirements for the project�Critical change requests�Intermediate change requests�Nice to have change requests
�Time spent on change requests up until a Y/Ndecision is given from the Senior Contractgroup
�Number and size of critical change requeststhat arise after the requirements phase hasbeen completed
-
Techniques for Meas Pgm - 29version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Set of StandardSet of StandardOrganizationalOrganizational
ProcessesProcesses
-
Techniques for Meas Pgm - 30version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Importance of anImportance of anOrganizational ViewOrganizational Viewof Processesof Processes
�Builds a common vocabulary�Allows others to anticipate behavior and be
more proactive in their interactions�Allows the organization to measure a
controlled set of processes to gain economy ofscale
�Trends can be seen and predictability can beachieved
�Process performance baselines can bedeveloped to support quantitativemanagement later
-
Techniques for Meas Pgm - 31version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
OrganizationalOrganizationalMeasurementMeasurementRepositoryRepository
�Develop an organization measurementrepository - include:�Product and process measures that are related to
the organization’s set of standard processes
�The related information needed to understandand interpret the measurement data and asses itfor reasonableness and applicability
�Develop operational definitions for themeasures to specify the point in the processwhere the data will be collected and for theprocedures for collecting valid data
-
Techniques for Meas Pgm - 32version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Slightly MoreSlightly MoreAdvanced MeasuresAdvanced Measures
-
Techniques for Meas Pgm - 33version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Peer ReviewsPeer Reviews
-
Techniques for Meas Pgm - 34version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Peer ReviewPeer ReviewMeasuresMeasures
�Optimum Checking Rate (e.g., Number ofpages to be checked per hour)
�Logging Rate (e.g., Number of major anddefects logged per hous)
�Number of Major and Minor Defects�Effectiveness - Number of Major Defects
found in this stage compared to the totalnumber of defects found so far
-
Techniques for Meas Pgm - 35version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
TestingTesting
-
Techniques for Meas Pgm - 36version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Defects DiscoveredDefects DiscoveredDuring TestingDuring Testing
�Effectiveness - Number of Major defects foundin a particular testing phase or instantiation ofa testing phase compared to the total numberof defects found during testing
�Number of defects projected to escape fromthe current testing phase
-
Techniques for Meas Pgm - 37version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Test CoverageTest Coverage
�Statement coverage measures whether eachexecutable statement is encountered
�Block coverage is the same as statementcoverage except that the unit of codemeasured is each sequence of non-branchingstatements
�Decision coverage measures whether booleanexpressions tested in control structures (suchas if-statements or while-statements)evaluated to both true and false
�Condition coverage measures the true or falseoutcome of each boolean sub-expression
-
Techniques for Meas Pgm - 38version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Test CoverageTest Coverage -- 22
�Function Coverage�Measures whether you invoked each function or
procedure�Call Coverage
�Measures whether you executed each function call�Based on the idea that faults commonly occur in
interfaces between modules�Loop Coverage
�Measures whether you executed each loop bodyzero times, exactly once, and more than once
�Relational Operator Coverage�Measures whether boundary situations occur with
relational operators (=)
-
Techniques for Meas Pgm - 39version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Risk ManagementRisk Management
-
Techniques for Meas Pgm - 40version CMMI Conf 2004© 2004 Kasse Initiatives, LLCImpact
Prob
abili
ty
Time
Prioritizing RisksPrioritizing Risks
�Priorities are established by merging theinformation available into a three dimensionaltable
-
Techniques for Meas Pgm - 41version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
PrioritizingPrioritizing -- 22
Catastrophic
Critical
Marginal
Negligible
Probable Possible Unlikely
Top
High
Serious
Medium
High
Serious
Medium
Low
Serious
Medium
Low
Bottom
Probability:
Impact:
-
Techniques for Meas Pgm - 42version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
PrioritizingPrioritizing -- 33
Immediate Short Medium
1
2
3
4
2
3
4
5
3
4
5
6
5 6 7
6 7 8
Top
High
Serious
Medium
Low
Bottom
Long
4
5
6
7
8
9
TimeFrame:
-
Techniques for Meas Pgm - 43version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Quantitative ProjectQuantitative ProjectManagementManagement
-
Techniques for Meas Pgm - 44version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
QuantitativeQuantitativeManagementManagementConceptsConcepts
�Quantitative Management is tied to theorganization’s strategic goals for productquality, service quality, and processperformance
�When higher degrees of quality andperformance are demanded, the organizationand projects must determine if they have theability to improve the necessary processes tosatisfy the increased demands
-
Techniques for Meas Pgm - 45version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
QuantitativeQuantitativeManagementManagementConceptsConcepts -- 22
�Achieving the necessary quality and processperformance objectives requires stabilizing theprocesses that contribute most to theachievement of the objectives
�Assuming the technical requirements can bemet, the next decision is to determine if it iscost effective
�Reducing process variation is an importantaspect to quantitative management
-
Techniques for Meas Pgm - 46version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Establish the Organization’sEstablish the Organization’sProcess PerformanceProcess PerformanceBaselineBaseline
�Process performance is a measure of theactual results achieved by following a process
�Measure the organization’s processperformance for the organization’s set ofstandard processes at various levelsincluding:�Individual processes (e.g., test case inspection
element)�Sequence of connected processes�Processes for the complete lifecycle�Processes for developing individual work products
-
Techniques for Meas Pgm - 47version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Establish the Organization’sEstablish the Organization’sProcess PerformanceProcess PerformanceBaselineBaseline -- 22
�There may be several process performancebaselines to characterize performance forsubgroups of the organization – Examplesinclude:�Product Line�Application domain�Complexity�Team size�Work product size
-
Techniques for Meas Pgm - 48version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Steps to Establish Project’sSteps to Establish Project’sQuality and ProcessQuality and ProcessPerformance ObjectivesPerformance Objectives
�Identify the quality and process performanceneeds and priorities of the customer, endusers, and other relevant stakeholders�Reliability�Maintainability and Expandability�Usability�Development cycle time
�Determine how quality and processperformance will be measured
-
Techniques for Meas Pgm - 49version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Steps to Establish Project’sSteps to Establish Project’sQuality and ProcessQuality and ProcessPerformance ObjectivesPerformance Objectives -- 22
�Define and document measurable quality andprocess performance objectives for the project
�Examples of Quality Objectives�Mean time between failures�Critical resource utilization
�Examples of Process Performance Objectives�Percentage of defects removed by type of
verification activity�Defect escape rates�Number and density of defects (by severity) found
during the first year following product delivery�Rework time as a percentage of total project life-
cycle time
-
Techniques for Meas Pgm - 50version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Steps to Establish Project’sSteps to Establish Project’sQuality and ProcessQuality and ProcessPerformance ObjectivesPerformance Objectives -- 33
�Establish objectives for each life-cycle stageto track progress and predict achievement ofproject’s objectives
�Establish criteria to identify whichsubprocesses are the main contributors toachieving the identified quality and processperformance objectives and for whichpredictable performance is important
-
Techniques for Meas Pgm - 51version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
MeasuresMeasuresand Analyticand AnalyticTechniquesTechniques
-
Techniques for Meas Pgm - 52version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Select Measures andSelect Measures andAnalytic TechniquesAnalytic Techniques
�Identify the measures that are appropriate forstatistical management
�Examples of subprocess control measuresinclude:�Ratios of estimated to measured values of the
planning parameters�Coverage and efficiency of work product
inspections�Test coverage and efficiency�Reliability, Maintainability, and Expandability�Percentage of the total defects inserted or found in
the different stages of the lifecycle
-
Techniques for Meas Pgm - 53version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
�There are a number of quantitative tools consideredto be applicable to statistical process or quality control:�Quantifying and Predicting Process Performance
� Control Charts� Histograms
�Cause and Effect Relationships� Scatter diagrams� Run charts� Cause-and-effect (fishbone) diagrams� Bar charts� Pareto charts� Interrelationship Diagraph� Kiviat Diagram
Quantitative ToolsQuantitative Tools
-
Techniques for Meas Pgm - 54version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Control ChartsControl Charts
�Control charts – techniques for quantifyingprocess behavior�Classical control charts have a centerline and
control limits on both sides of the centerline�Both the centerline and the limits represent
estimates that are calculated from a set ofobservations collected while the process is running
�The centerline and control limits cannot beassigned arbitrarily as they are intended to showwhat the process can actually do
Florac, W.A. & Carleton, A.D. Measuring the Software ProcessAddison-Wesley, 1999
-
Techniques for Meas Pgm - 55version CMMI Conf 2004© 2004 Kasse Initiatives, LLCNumerical data taken
in time sequence
UpperControlLimit(UCL)
LowerControlLimit(LCL)
Upper andLower
Control Limitsrepresent the
natural variationIn the process
METRIC:PROCESS CONTROL CHART TYPE:
A point above or below thecontrol lines suggests that the
measurement has a specialpreventable or removable cause
Plotted points are eitherindividual measurements or the
means of small groups ofmeasurements
The chart is used for continuousand time control of the process
and prevention of causes
The chart is analyzed usingstandard Rules to define the
control status of the process
Datarelating to
the process
Center Line (CL)(Mean of data used toset up the chart)
Statistical Methods for Software QualityAdrian Burr – Mal Owen, 1996
-
Techniques for Meas Pgm - 56version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
HistogramsHistograms
�Histograms�Allows a project team to take measurement data
and display the distribution of the observed values�Show the frequencies of events that have occurred
in ways that make it easy to compare distributionsand see central tendencies
�Illustrates quickly the underlying distribution of thedata
�Helps indicate if there has been a change in theprocess
Florac, W.A. & Carleton, A.D. Measuring the Software ProcessAddison-Wesley, 1999
-
Techniques for Meas Pgm - 57version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
HistogramsHistograms -- 22
�Histograms - continued�Provides useful information for predicting future
performance of the process�Helps answer the question “Is the process capable
of meeting my customers requirements?”�Histograms are created by grouping the results into
“cells” and then counting the number in each cell
Florac, W.A. & Carleton, A.D. Measuring the Software ProcessAddison-Wesley, 1999
-
Techniques for Meas Pgm - 58version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Num
ber o
f Day
s
Product – Service Staff Hours
20
56
181614121086420
32 5452504846444240383634
HistogramsHistograms
-
Techniques for Meas Pgm - 59version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Applying StatisticalApplying StatisticalMethods toMethods toUnderstandUnderstand
VariationVariation
-
Techniques for Meas Pgm - 60version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Understanding VariationUnderstanding Variation
�Understanding variation is achieved bycollecting and analyzing process and productmeasures so that special causes of variationcan be identified and addressed to achievepredictable performance
�All characteristics of processes and productsdisplay variation when measured over time
�Variation may be due to�Special or “assignable” causes of variation�Natural or common causes
-
Techniques for Meas Pgm - 61version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
SummarySummary
�Techniques for Establishing a MeasurementProgram include:�Clearly define the need for a measurement
program�Establish a measurement initiative with objectives
that are aligned with established information needsand business objectives
�Ensure basic measures are included for planning,tracking, and taking corrective action as necessary
�Incorporate process effectiveness measures�Establish organizational standard processes
-
Techniques for Meas Pgm - 62version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
SummarySummary -- 22
�Establish and utilize measures such as peer reviewmeasures, testing measures, and risk managementmeasures
�Evolve into project management based on aquantitative understanding of the organization’sand project’s defined processes
-
Techniques for Meas Pgm - 63version CMMI Conf 2004© 2004 Kasse Initiatives, LLC
Tim KasseTim KasseKasse Initiatives LLCKasse Initiatives LLC
PMB 293PMB 2931900 Preston Road, Suite 2671900 Preston Road, Suite 267
Plano, Texas 75093Plano, Texas 75093U.S.A.U.S.A.
Tel: +1 972 987 7606Tel: +1 972 987 7606
E-mail: [email protected]
Web Site: www.kasseinitiatives.com
Evolving a Measurement Program f...WelcomeMeasurement�Is It Really Necessa...At Project Start, Do You Know…?Can it be done?How long will it take?How much will it cost?How many people will it take?What is the risk?What are the tradeoffs?How many errors will there be?
What Do You Know Now?How much does your current devel...How much value does each piece o...What would the impact be of dele...What activities contribute the m...Have you tried to improve the cu...What changes in cost/value resul...
What Do You Know Now? - 2What processes represent the gre...How would you quantify the value...Do you really want to know where...What value do you think you are ...How much is the knowledge of you...
Measurement �and AnalysisMeasurement and Analysis Overvie...A measurement initiative involve...Specifying the objectives of mea...Defining the measures to be used...Implementing the collection, sto...Providing objective results that...
Establish Measurement ObjectivesMeasurement objectivesDocument the purposes for which ...Specify the kinds of actions tha...Continually ask the question – w...Involve the end users of the mea...Maintain traceability of the pro...
Specify MeasuresIdentify candidate measures base...Define how data can and will be ...Examples of Commonly Used Derive...Defect densityPeer review coverageReliability measures
Specify Data Collection and Stor...Specify how to collect and store...Make explicit specifications of ...Develop procedures for ensuring ...Ensure that the data is stored s...
Specify Analysis ProceduresDefine the analysis procedures i...Ensure that the results that wil...Collecting data for the sake of ...Showing how it can be used to ma...
Collect �Measurement DataCollect the measurement data as ...Generate data for derived measur...Perform integrity checks as clos...
Analyze the Measurement DataConduct the initial analysesInterpret the results and make p...Conduct additional measurement a...Review the initial results with ...Prevents misunderstandings and r...Improve measurement definitions,...
Store the Measurement Data and A...The stored information should co...Understand the measuresAssess them for reasonableness a...The stored information should al...Enable the timely and cost effec...Provide sufficient context for i...
Communicate the Measurement Resu...Keep the relevant stakeholders u...Follow up with those who need to...Increases the likelihood that th...Assist the relevant stakeholders...
Measurement and Analysis GroupConsider creating a measurement ...
Measurement and Analysis ToolsIncorporate tools used in perfor...Statistical packagesDatabase packagesSpreadsheet programsGraphical packagesPackages that support data colle...
Measurement and Analysis Trainin...Provide training to all people w...Data collection, analyses, and r...Measurement toolsGoal-Question-Metric ParadigmHow to establish measureshow to determine efficiency and ...Quality factors measures (e.g., ...Basic and advanced statistical t...
Basic MeasuresBasic MeasuresEstimate Size and/or Complexity ...Examples of attributes to estima...Number of logic gatesNumber of interfacesExamples of size measurements fo...Function PointsLines of CodeNumber of requirements
Basic Measures - 2Determine effort and cost – use ...Establish the project’s schedule...Include, or at least consider, i...Identify risks associated with t...Control data (various forms of d...
Basic Measures – 3Identify the knowledge and skill...Select and implement methods for...Training (Internal and External)MentoringCoachingOn-the-job application of learne...Monitor staffing needs – based o...
Basic Measures - 4Involve relevant stakeholders th...Track technical performance (Com...Product components designed, con...Compare actual milestones comple...Monitor commitments and critical...
Basic Measures - 5Track quality – Problems/Defects...Problems and defects are direct ...
Effectiveness of ProcessesEffectiveness of ProcessesIn addition to defining the proc...How well are the processes worki...
Requirements Management – Effect...Number of change requests per mo...Critical change requestsIntermediate change requestsNice to have change requestsTime spent on change requests up...Number and size of critical chan...
Set of Standard Organizational P...Importance of an Organizational ...Builds a common vocabularyAllows others to anticipate beha...Allows the organization to measu...Trends can be seen and predictab...Process performance baselines ca...
Organizational Measurement Repos...Develop an organization measurem...Product and process measures tha...The related information needed t...Develop operational definitions ...
Slightly More Advanced MeasuresPeer ReviewsPeer Review�MeasuresOptimum Checking Rate (e.g., Num...Logging Rate (e.g., Number of ma...Number of Major and Minor Defect...Effectiveness - Number of Major...
TestingDefects Discovered During Testin...Effectiveness - Number of Major ...Number of defects projected to e...
Test CoverageStatement coverage measures whet...Block coverage is the same as st...Decision coverage measures wheth...Condition coverage measures the ...
Test Coverage - 2Function CoverageMeasures whether you invoked eac...Call CoverageMeasures whether you executed ea...Based on the idea that faults co...Loop CoverageMeasures whether you executed ea...Relational Operator CoverageMeasures whether boundary situat...
Risk ManagementPrioritizing RisksPriorities are established by me...
Prioritizing - 2Prioritizing - 3Quantitative Project ManagementQuantitative Management ConceptsQuantitative Management is tied ...When higher degrees of quality a...
Quantitative Management Concepts...Achieving the necessary quality ...Assuming the technical requireme...Reducing process variation is an...
Establish the Organization’s Pro...Process performance is a measure...Measure the organization’s proce...Individual processes (e.g., test...Sequence of connected processesProcesses for the complete lifec...Processes for developing individ...
Establish the Organization’s Pro...There may be several process per...Product LineApplication domainComplexityTeam sizeWork product size
Steps to Establish Project’s Qua...Identify the quality and process...ReliabilityMaintainability and Expandabilit...UsabilityDevelopment cycle timeDetermine how quality and proces...
Steps to Establish Project’s Qua...Define and document measurable q...Examples of Quality ObjectivesMean time between failuresCritical resource utilizationExamples of Process Performance ...Percentage of defects removed by...Defect escape ratesNumber and density of defects (b...Rework time as a percentage of t...
Steps to Establish Project’s Qua...Establish objectives for each li...Establish criteria to identify w...
Measures �and Analytic Technique...Select Measures and Analytic Tec...Identify the measures that are a...Examples of subprocess control m...Ratios of estimated to measured ...Coverage and efficiency of work ...Test coverage and efficiencyReliability, Maintainability, an...Percentage of the total defects ...
Quantitative ToolsThere are a number of quantitati...Quantifying and Predicting Proce...Control ChartsHistogramsCause and Effect RelationshipsScatter diagramsRun chartsCause-and-effect (fishbone) diag...Bar chartsPareto chartsInterrelationship DiagraphKiviat Diagram
Control ChartsControl charts – techniques for ...Classical control charts have a ...Both the centerline and the limi...The centerline and control limit...
Slide 55HistogramsHistogramsAllows a project team to take me...Show the frequencies of events t...Illustrates quickly the underlyi...Helps indicate if there has been...
Histograms - 2Histograms - continuedProvides useful information for ...Helps answer the question “Is th...Histograms are created by groupi...
HistogramsApplying Statistical Methods to ...Understanding VariationUnderstanding variation is achie...All characteristics of processes...Variation may be due toSpecial or “assignable” causes o...Natural or common causes
SummaryTechniques for Establishing a Me...Clearly define the need for a me...Establish a measurement initiati...Ensure basic measures are includ...Incorporate process effectivenes...Establish organizational standar...
Summary - 2Establish and utilize measures s...Evolve into project management b...
Tim Kasse�Kasse Initiatives LLC�...