high maturity workshop @eurospi...

HIGH MATURITY WORKSHOP @EUROSPI CONFERENCEChristian Hertneck

June 2014

> Motivation

> Statistical Thinking

> Capability and Maturity Level 4 & 5

> Overview of Statistical Tools

> Workshop Topics> Examples

> Exercises

> Discussions

> Workgroups

2

CONTENT

4

TYPICAL REPORTING SYSTEM

QualityJuly

Actual Value

Monthly Average

Value% Diff % Diff from

July Last Yr Actual Plan or Average % Diff

This YTD as % of

Last YTDOn-time Shipments (%) 91.0 91.3 -0.3 -0.9 90.8 91.3 -0.6 -0.3First Time Approval (%) 54.0 70.0 -23.0 -10.0 69.3 70.0 -1.0 -0.4Pounds Scrapped (per 1000 lbs production) 124.0 129.0 -3.9 0.0 132.0 129.0 2.3 1.5

Production

Production Volume (100s/lbs) 34.5 36.0 -4.2 -2.0 251.5 252.0 -0.2 -8.0Material Costs ($/100 lbs) 198.29 201.22 -1.5 -1.9 198.5 201.2 -1.4 -3.6Manhours per 100 lbs 4.45 4.16 7.0 4.5 4.5 4.2 7.2 9.3Energy & Fixed Costs/100 lbs 11.34 11.27 0.6 11.3 11.02 11.27 -2.2 9.2Total Production Costs/100 lbs 280.83 278.82 0.7 0.9 280.82 278.82 0.7 0.4In-process Inventory (100's lbs) 28.00 19.70 + 42.0 + 12.0 21.6 19.7 + 9.6 + 5.9

Yr-to-Date

Monthly Report

5

TIME SERIES FOR MONTHLY IN-PROCESS INVENTORY

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov DecYr 1 19 27 20 16 18 25 22 24 17 25 15 17Yr 2 20 22 19 16 22 19 25 22 18 20 16 17Yr 3 20 15 27 25 17 19 28

In-Process Inventory

5

10

15

20

25

30

35

J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J

20.04

Average

No long term trendsNo other systematic patterns

Does not say whether July value is exceptional Is the July value a signal---or is it just noise?

In-process Inventory (100's lbs) 28.00 19.70 + 42.0 + 12.0 21.6 19.7 + 9.6 + 5.9

July Actual Value

Monthly Average

Value% Diff

% Diff from July Last Yr

ActualPlan or

Average% Diff

This YTD as % of

Last YTD

Yr-to-Date

Need to filter out the month to month variation.

28

6

0

5

10

15

J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J

4.35

14.2 = upper limit for difference between Monthly values

Moving range directly measures the month-to-month variation.Upper control limit for Moving Range = 3.27 x Avg = 14.2

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec8 7 4 2 7 3 2 7 8 10 2

3 2 3 3 6 3 6 3 4 2 4 13 5 12 2 8 2 9

Result: If the amount of in-process inventory changes (up or down) by more than 1420 lbs from one month to the next, then one should look for an explanation.

UPPER CONTROL LIMIT FOR MOVING RANGE CHART

Result of monitoring the process behavior chart: No problem! 7

TIME SERIES FOR MONTHLY IN-PROCESS INVENTORY

Size of % difference partially depends upon the magnitude of the base number10 unit change from 100 to 110 is 10% change10 unit change from 300 to 310 is 3.3% change

Comparing lines in a monthly report by comparing the size of the percent differences assumes that all lines should show the same amount of relative variation month-to-month

Differences based upon comparison of the current value with last year’s value, a large % difference may be due to an unusual value in the past rather than an unusual value in the present

8

PROBLEMS WITH PERCENT DIFFERENCES AS A BASIS FOR INTERPRETING VALUES

Month-to-month variation

Inventory change > 1420 lbs from one month to the next, one should look for an explanation…likely due to the direct result of some special / assignable cause.

Actual Values Monthly Chart

Limits on the individual values chart define how large or small a single monthly value must be before it represents a definite departure from the historic average…value in excess of 31.6 would be a signal that the amount of inventory had shifted upward.

Value of 28 is not, by itself, a signal. There is not real

evidence of any real change in the in-process inventory.

9

INTERPRETATION

10

QualityJuly

Actual Value

Monthly Average

Value% Diff % Diff from

July Last Yr Actual Plan or Average % Diff

This YTD as % of

Last YTDOn-time Shipments (%) 91.0 91.3 -0.3 -0.9 90.8 91.3 -0.6 -0.3First Time Approval (%) 54.0 70.0 -23.0 -10.0 69.3 70.0 -1.0 -0.4Pounds Scrapped (per 1000 lbs production) 124.0 129.0 -3.9 0.0 132.0 129.0 2.3 1.5

Production

Production Volume (100s/lbs) 34.5 36.0 -4.2 -2.0 251.5 252.0 -0.2 -8.0Material Costs ($/100 lbs) 198.29 201.22 -1.5 -1.9 198.5 201.2 -1.4 -3.6Manhours per 100 lbs 4.45 4.16 7.0 4.5 4.5 4.2 7.2 9.3Energy & Fixed Costs/100 lbs 11.34 11.27 0.6 11.3 11.02 11.27 -2.2 9.2Total Production Costs/100 lbs 280.83 278.82 0.7 0.9 280.82 278.82 0.7 0.4In-process Inventory (100's lbs) 28.00 19.70 + 42.0 + 12.0 21.6 19.7 + 9.6 + 5.9

Yr-to-Date

Monthly Report

TYPICAL REPORTING SYSTEM

11

TIME SERIES FOR PERCENTAGE ON-TIME SHIPMENTS

12

INTERPRETATION

Six of the individual values and one of the moving ranges fall outside the limits.

The six values below the limit shouldbe treated as signals. The process is trying totell you it has a problem.

If you concentrate on the percent differences in the monthly report, you are not likely to ever be aware of this problem until it is already too late…

Control Charts are about separating signals from noise.

> Motivation





> Exercises

> Discussions

> Workgroups

13

CONTENT

14

PROCESS VARIATION

Shewhart’s notion of dividing variation into two types:

> common cause variation > variation in process performance due to normal or

inherent interaction among process components (people, machines, material, environment, and methods)

> represents the “noise” of the process

> assignable/special cause variation > variation in process performance due to events that are not part of the

normal process. > represents sudden or persistent abnormal changes in one or more of the

process components> represents a “signal” of the process

15

COMMON CAUSES

16

BY CHANGING THE PROCESS YOU HAVE TO ADAPT THE CONTROL CHARTS

17

PREDICTABILITY IS THE KEY FOR HIGH MATURITY

> A process is said to be predictable when, through the use of past experience, we can describe, at least within limits, how the process will behave in future.

> A unpredictable process will display exceptional variation that is the result of assignable/special causes.

> A predictable process will display routine variation that is characteristic of common causes.

Image courtesy of Microsoft Cliparts

18

REMOVING SPECIAL CAUSES OF VARIATION

> Difficult to do well!> Identify special/assignable causes as soon as possible

> checks/alerts/triggers in data entry/measurement tools> checks within the measured process> use charts/diagrams for daily review> assign individuals to monitor process data> provide template/form to fill out initial details

of special variance> Maintain data base of special cause investigations> Use data base reports to monitor corrective actions> Train individuals for the investigations and problem

solving techniques.

19

IDENTIFYING SPECIAL CAUSES OF VARIATION - EXAMPLE

20

FROM PROCESS STABILITY TO PROCESS PREDICTABILITY

Process is predictable!

Process is stable!

Derive prediction model based on an appropriate*probability model

appropriate* means the model fits the reality

Determine stability of the process

21

Managing Quantitatively? - Example

Therefore, just having nice charts and diagrams DOES NOT mean you are statistically controlling anything.

22

SIMPLE DETECTION TESTS FOR INSTABILITIES

23

REMOVING SPECIAL CAUSES OF VARIATION - REVISITED

> Use traditional, effective, problem solving techniques to select corrective action

> team of experts> addressing root causes of problems> obtain approval, pilot, deploy, train, monitor, inform

> Beware of “false alarms”> data errors, inconsistencies> improperly calculated limits, wrong assumptions> lack of data grouping> too many tests> nothing wrong with the process

> Beware of a lack of any special variation> control limits may be too wide

> Improving data quality and data grouping is a continuous effort

24

LISTENING TO VOICES

Process Capability

DOES NOT EQUATE TO a capable process.

Voice of the process = the natural bounds of process performance

Stable

UCL

LCL

Capable

UCL LCL

USL

LSL

USL/LSL = Upper/Lower Specification LimitUCL/LCL = Upper/Lower Control Limit

• Capable process = stable process + product conformance

• Voice of the customer = the goals established for the product and process performance (e.g. specifications)

USL

LSL

25

ALIGNING PROCESS PERFORMANCE TO PROCESS REQUIREMENTS

Change the Specs

Upper Spec

Shift the AimUpper Spec

Mean

Reduce VariationUpper Spec

> Motivation





> Exercises

> Discussions

> Workgroups

26

CONTENT

PA 4.1 Process measurement attribute> Measure of the extent to which measurement results are used to

ensure that performance of the process supports the achievement of relevant process performance objectives in support of defined business goals.

> Result of full achievement> process information needs in support of relevant defined business goals are

established;> process measurement objectives are derived from process information needs;> quantitative objectives for process performance in support of

relevant business goals are established;> measures and frequency of measurement are identified and

defined in line with process measurement objectives and quantitative objectives for process performance;

> results of measurement are collected, analyzed and reported in order to monitor the extent to which the quantitative objectives for process performance are met;

> measurement results are used to characterize process performance. ISO/IEC 15504-2

CAPABILITY LEVELS 4 & 5 (SPICE)

0 Incomplete

5 Optimizing

4 Predictable

3 Established

2 Managed

1 Performed

27

PA 4.2 Process control attribute

> Measure of the extent to which the process is quantitatively managed to produce a process that is stable, capable, and predictable within defined limits.

> Result of full achievement> analysis and control techniques are determined and applied where applicable;> control limits of variation are established for normal process performance;> measurement data are analyzed for special causes of variation;> corrective actions are taken to address special causes of variation;> control limits are re-established (as necessary) following corrective action.

ISO/IEC 15504-2


0 Incomplete

5 Optimizing

4 Predictable

3 Established

2 Managed

1 Performed

28

PA 5.1 Process innovation attribute> Measure of the extent to which changes to the process are identified

from analysis of common causes of variation in performance, and from investigations of innovative approaches to the definition and deployment of the process.

> Result of full achievement> process improvement objectives for the process are defined that support the

relevant business goals;> appropriate data are analyzed to identify common causes of variations in process

performance;> appropriate data are analyzed to identify opportunities for best practice and

innovation;> improvement opportunities derived from new technologies and process concepts

are identified;> an implementation strategy is established to achieve the process improvement

objectives.


0 Incomplete

5 Optimizing

4 Predictable

3 Established

2 Managed

1 Performed

29

PA 5.2 Process optimization attribute

> Measure of the extent to which changes to the definition, management and performance of the process result in effective impact that achieves the relevant process improvement objectives.

> Result of full achievement> impact of all proposed changes is assessed against the objectives of the defined

process and standard process;

> implementation of all agreed changes is managed to ensure that any disruption to the process performance is understood and acted upon;

> effectiveness of process change on the basis of actual performance is evaluated against the defined product requirements and process objectives to determine whether results are due to common or special causes.

ISO/IEC 15504-2


0 Incomplete

5 Optimizing

4 Predictable

3 Established

2 Managed

1 Performed

30

CMMI – CONNECTING CAPABILITY AND MATURITY ON ORGANIZATIONAL LEVEL

Maturity Level 5Optimizing

Organizational Performance ManagementCausal Analysis and Resolution

Capability Level 1 2 3

Maturity Level 4Quant. Managed

MaturityLevel

3Defined

Organizational Process PerformanceQuantitative Project Management

Requirements Management Project PlanningProject Monitoring and ControlSupplier Agreement ManagementMeasurement and AnalysisProcess and Product Quality AssuranceConfiguration Management

Requirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidationOrganizational Process FocusOrganizational Process DefinitionOrganizational Training Integrated Project ManagementRisk ManagementDecision Analysis and Resolution

MaturityLevel

2Managed

31

32

WHAT AN ORGANIZATIONAL LEVEL 3 HAS ACHIEVED …

» Managing a company by means of the monthly report is like trying to drive a car by watching the yellow line in the rear-view mirror «

[Myron Tribus]

33

HIGH MATURITY REQUIRES LEADING INDICATORS

34

SUMMARY OF TERMINOLOGY

Improve process continually to reduce variability and improve quality, cost and cycle time.

Improvement

Capability

Predictability

Stability

Performance

Ensure that process behaviour is stable by removing assignable causes.

Make the process capable by changing the process performance.

Focu

s of

Lev

el 4

A process is said to be predictable when through the use of past experience, you can describe, at least within limits, how the process will behave in future.

Focu

s of

Lev

el 5

Measuring of attributes of the process

Focu

s of

ML

3

> Motivation





> Exercises

> Discussions

> Workgroups

35

CONTENT

36

PURPOSE AND BENEFITS OF SPC

„Provide quantitative information that improves decision making in time to positively affect the business outcome

Decisions regarding adaptations may be necessary due to changing circumstances on project, product, process, or business level

Suitable information is necessary to be able to decide what needs adaptation on business, process, or project level Early identification of critical project situations and problems

Early identification of critical project situations and problemsObtain leading (measures to decide) instead of lagging

(measures to learn) indicatorsConsistent prediction and monitoring across different levels

(project, multi-project, organizational)

37

WHAT IS STATISTICAL PROCESS CONTROL?

Typical questions to be answered using SPC

Is process x in control and predictable?(Is the quality of work products generated in the design phase predictable?)

What is the average value and range for process x?(How many hours do we typically spend on peer reviews of design documents?)

Is the latest measurement typical or did something unusual happen (need for corrective action)?

SPC is an analytical / statistical technique used to identify and understand sources of process performance variations in order to quantitatively monitor, control, and predict the process.

QUANTITATIVE/STATISTICAL TOOLS AND TECHNIQUES - 1

Statistics

Inferential Statistics Descriptive StatisticsPrecondition

Watching Statistics monitor variation, i.e. distinguish between usual random

and abnormal change.

Inferential statistics comprises the use of statistics to make inferencesconcerning some unknown aspect (usually a parameter) of a population

Descriptive statistics is a branch of statisticsthat denotes any of the many techniques used to summarize a set of data. In a sense, we are usingthe data on members of a set to describe the set.

Watching statistics are applied to assess the nature of variation in a process and to facilitate forecasting and management (monitor variation, i.e. distinguish between usual random from abnormal change).

39

QUANTITATIVE/STATISTICAL TOOLS AND TECHNIQUES - 2

Statistics

Inferential Statistics Descriptive StatisticsPrecondition

Numerical Graphical

Mean MedianMode

Pareto ChartHistogramRun ChartContingency T.Scatter PlotFishboneBox Plot

Location Dispersion

Range, IQRStandard DeviationVariance

Distributions(the shape of the process)

Hypothesis Testing

Regression Analysis

Prediction Models

Parameter Estimation

Watching Statistics monitor variation, i.e. distinguish between usual random from abnormal change.

> If we measure more things, and involve more people in reviewing and using the measures, we will eventually achieve High Maturity...

> The key to achieving high maturity is measuring the right things, and using the correct techniques to analyze and interpret the measures...

> We need to wait until we have more of the right kind of data before we can attempt to implement High Maturity Practices...

40

MISPERCEPTIONS ABOUT HIGH MATURITY - 1


> Adding the use of Control Charts to the practice of measurement and analysis results in High Maturity…

> All I need to do is to use Control Charts to analyze the outcome of our critical subprocesses and we can control them...

> Using threshold based on specification limits is a high maturity practice..

41



> All the things we need to understand about high maturity practices can be adequately explained in a single conference presentation or workshop. [10]

42



43

Clarify business goals

Identify and prioritize issues

Select and define measures

Collect, verify, and retain data

Analyze process behaviour

Processstable?

Processcapable?

New goals,strategy?

New issues?

Newmeasures?

Remove assignable causes

Change process

Continually improve

Yes

Yes

YesNo

No

No

No

No

Yes

Yes

Leve

l 5

Leve

l 4

Leve

l 3

SUMMARY –PROCESS BEHAVIOR MEASUREMENT FRAMEWORK

44

HIGH MATURITY IS ABOUT IDENTIFYING WASTE AND IMPROVEMENT OPPORTUNITIES QUANTITATIVELY

> Motivation





> Exercises

> Discussions

> Workgroups

45

CONTENT

> High Maturity case studies

> Selecting processes and data for statistical analysis

> Connecting business improvement to quantitative data

> Creating performance models and their use in project management.

> Tools for use in statistical analysis of data

46

SAMPLES FOR WORKGROUP DISCUSSIONS

47

QUESTIONS

THANK Y0U!

Christian Hertneck Anywhere.24 GmbHLindberghstr. 1182178 Puchheimwww.anywhere24.com

VersionDraft /

for review / released

Date Comments/Change History Author

0.01 Draft 06.02.2014 Layout; Contents; Summary... C.Hertneck

0.02 For review 17.06.2014 Contents C.Hertneck

1.00 Released 23.06.2014 Final touches C.Hertneck

® Capability Maturity Model, Carnegie Mellon, CMM, andCMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

sm CMM Integration; IDEAL; Personal Software Process; PSP; SCAMPI; SCAMPI Lead Assessor/ Appraiser; SEPG; Team Software Process; and TSP are service marks of Carnegie Mellon University.

> [1] Measuring the Software Process, William Florac and Anita Carleton, Addison-Wesley, 1999.

> [2] Understanding Variation: Key to Managing Chaos, Donald Wheeler, SPC Press, 1993.

> [3] Understanding Statistical Process Control, Donald Wheeler and David Chambers, SPC Press, 1992.

> [4] Practical Software Metrics for Project Management and Process Improvement, Robert Grady, Englewood Cliffs, 1992.

> [5] Statistical Methods for Software Quality - Using Metrics to Control Process and Product Quality, Adrian Burr and Mal Owen, International Thomson Computing Press.

> [6] Goal-Driven Software Measurement - A Guidebook, Robert Park, WolfhartGoethert and William Florac, CMU/SEI-96-HB-002, Carnegie Mellon University.

> [7] Metrics and Models in Software Quality Engineering, Stephen Kan, Addison Wesley, 1995.

> [8] Software Metrics: A Rigorous & Practical Approach, Norman Fenton, Shari Pfleeger, Thomson, 1997.

> [9] Capability Maturity Model® Integration (CMMI-DEV v1.3).

> [10 ]SEPG Presentations, e.g., High Maturity Misconceptions, Will Hayes, 2007

51

REFERENCES

high maturity workshop @eurospi...

Documents