SPI – past and future

Tor StålhaneIDU / NTNU

What has happened – 1 Most of the SPI that was done before 1980 – 1990 was based on •the work of Deming on TQM (Total Quality Management)•ideas from DoE (Design of Experiments), with a scientific basis using statistical analysis.

GQM from the University of Carnegie Mellon in cooperation with NASA, had a large impact. GQM focuses on:•Data collection based on scientific principles•Data analysis based on statistical methods

What has happened – 2 The need for sound scientific results was driven by the ambition of arriving at general results.This led to the use of statistical analyses with a high level of significance – e.g. 95% – which again led to the need for a large amount of data, which •was difficult for many small organizations.•led to the need for collecting data over a long time if the organization did not have a large project.•led to a long lead time, which made the improvement process slow. As long as the introduction of new tools, methods and processes was slow, this was not a serious problem.

The late 90’s – 1

The main problems were the •amount of data needed in order to use statistical analysis•calendar time needed to collect the dataThe SPI community changed to qualitative methods•not founded in traditional science•founded in business and social sciences, e.g. force field diagrams, affinity diagrams and root-cause analysis.

The late 90’s – 2 The qualitative methods from business and social sciences are •simple•rely mostly on subjective data picked from the participants’ knowledge and memory•consume a considerable amount of resources.

A post mortem analyses in a five person project will•consume a person-week to collect and analyse the data.•not allow you to identify any effect of an improvement until the end of the next project.

Agile software developmentand SPI

What’s new – 1?SPI will by its very nature always contain the

activities:• Identify things that we want to improve• Decide how to improve them• Implement the necessary changesWhat varies is:• When we do improvement• How we identify what we want to improve• How we make decisions

What’s new – 2?The main thing with agile SPI is the opportunity offered by

splitting the project into several short iterations.Traditional software development: • Run project• Discover problems• Suggest and implement improvements• Try out in the next projectAgile software development• Run an iteration • Discover problems• Suggest and implement improvements• Try out in next iteration

Agile SPI – 1

The current problems made ideas from agile development appealing. A partly agile approach called Rapid Process Improvement (RPI) was suggested by I. Seward et al. and presented at an SPI conference in Gothenburg in 2000. The Scandinavian firm Tieto-Enator has developed a process improvement method with focus on checklists and short iterations.

Agile SPI – 2 The current focus is not on collecting and analysing large amounts of data. We have two competing concepts. •Data-heavy concepts such as – QIP (Quality Improvement Paradigm) which uses GQM

(Goal Question Metrics) to collect and analyse large amounts of data

– TQM (Total Quality Management)•Scrum's sprint retrospectives.

In between these two extremes there is plenty of room for concepts like checklist driven SPI, post mortem analysis and other, related ideas.

Agile SPI – 3 The principles of agile development request that we:•at regular intervals reflects over how to become more effective•tunes and adjusts our behaviour

The focus is on •the need for SPI within the on-going project•moving process control from the organization level to the practitioners •the immediate use of experiences of the developers in improving the on-going project

Agile SPI – 4Expected benefits are e.g.:•Better collaboration between SPI project and developers•Better able to adopt changes•Quicker deployment of new processThe emphasis is on face-to-face communication as the primary means for conveying information•To the development team•Within the development team

Agility and communication

The most important reason why agile development has been a success is that it improves communication.

For software development – and for many other areas – bad communication is the root of all evil.

Communication efficiency – 1

Communication efficiency – 2

Communication Strategy Within Team With Stakeholders

Face to face (F2F) 4.25 4.06 F2F at Whiteboard 4.24 3.46 Overview diagrams 2.54 1.89 Online chat 2.10 0.15 Overview documentation 1.84 1.86

Teleconference calls 1.42 1.51 Videoconferencing 1.34 1.62 Email 1.08 1.32 Detailed documentation -0.34 0.16

Communication efficiency – 3 • Physical proximity. The closer people are to one

another the greater the opportunities for communication. At one end of the spectrum two people can be working side-by-side and at the other end of the spectrum two people can be in different buildings.

• Temporal proximity. Whether or not two people are working at the same time affects communication.

• Amicability. Cockburn believes that amicability is an important success factor. Greater amicability =>• greater amount and quality of information will be

communicated• less will be concealed.

Communication efficiency – 4

Nothing beats face-to-face communication – preferably supported by a white board.

Nothing is worse that communication through large amounts of detailed documents

For many companies, however, the preferred way of communicating with customers and with assessors is through detailed documents, which is the least effective way of communicating.

Challenges Many agile methodologies suggest iterative SPI as an important part of development but lack:•Specific, clear procedures for this activity•Focus and long duration of perspective•Means for validating the implemented process improvement •Systematic follow-up of improvement activities

SPI and Scrum – 1 Scrum has an explicit SPI activity at the end of each sprint. Most of the material on Scrum SPI is found in blogs. Some important points from one of the bloggers: •Establish trust so that everybody feel free to speak his mind. •Identify the facts – data collection, one way or another. •Identify the team's strengths – what do we do well•Collect ideas for improvement – where can we improve ourselves and how•Consolidate and prioritize the ideas.

SPI and Scrum – 2 Small datasets in agile SPI may result in improvements that are not as good as the ones achieved using e.g. GQM. The agile paradigm’s strong points:•No preparations – e.g., courses – are needed.•The time span we consider in the retrospective – one sprint – is usually short and the participants remember all important facts and events – no data collection is needed during development.•The effect of any SPI action that we implement can be observed in the next sprint.•We do these SPI activities for this project – quick pay-back.

Benefits of agile SPI According to Ericsson, the befits of agile SPI are•Higher commitment than in traditional SPI•Iterations give quick feedback – easier to adopt changes•Increased awareness of the value of process improvement– Which processes are needed and why– What is expected from the process

Problems for agile SPI

The main problems:•Focus on this project – short sighted SPI•No mechanism for transferring the improvement to the rest of the company•No defined method for checking that the changes have – the intended effects– no unintended side-effects

Cost and risk – 1 Cost and risk are related and they spell out the ambition of the SPI activity. If we are •going for scientific and general results => large amounts of data and a statistical analysis. •satisfied with a quick answer which might apply to only the problem at hand=> just a few data and a simple analysis method. We can reduce the risk of a wrong decision by •collecting more data.•take smaller steps On the other hand, postponing the decision of implementing a new and presumably better method and using time for data collections also have their costs.

Expert knowledge vs. data

Use of collected data

Use of expert knowledge

Risk

Cost and risk – 2

We can reduce the risks by doing small improvement steps. In this way, we can move in small steps with a low risk but also with small effects. The small effects will add up and, in the end, give large improvement.

This is for instance the main idea of RPI.

Time frame and stability Environmentstability

Company time frame

Risk management

Mean and lean

Measurementbased SPI

Short Medium Long

High

Medium

Low

CMM

We are moving this way

Clear trends – 1 • More and more companies will adapt all or a considerable

part of Scrum. • Availability of free-ware tools will increase the speed of

adaptation of new tools. It is easy to try new ways to solve old problems. The investment is small – learn the tool and try it out – and the benefits can be large.

• For many application areas, adaptability and flexibility are more important than reliability. It is easy to roll back to a previous version if we make mistakes.

• For several application areas, software development is the bottleneck when deploying a new product. This is for instance the case for cars. There will be increased pressure on the software industry to deliver faster.

Clear trends – 2 One trend might overshadow the others – outsourcing. •Many Norwegian software development companies are under considerable price pressure. This has lead to outsourcing – first to India and then to Eastern Europe, e.g., Ukraine and the Baltic states. •This lead to a situation where the Norwegian company is responsible for customer contact and high-level design•Development and testing are done through outsourcing to a separate company, a foreign branch or to a foreign partner. The question of SPI will then be concerned with how to select the best outsourcing company or partner.