10 Industrial Management

Predictive analytics boosts product development

EXECUTIVE SUMMARYManufacturing companies

in high-wage countries strive to

shorten development and innovation

cycles and decrease costs to strengthen

their competitive position. Efficient and

lean development projects can help. This article

presents the concept of using predictive analytics

to anticipate deviations – and thus inefficiencies

– from the development project’s target process.

Adapting the hot spot analysis that is used in

predictive crime analysis allows companies to

come up with new products more efficiently.

BY STEFAN RUDOLF AND CHRISTIAN DOELLE

january/february 2016 11

The deviation

probability map identifies

hot spots, where

problems are likely to occur.

Globalization has created a number of challenges for manufacturing companies, including shortened product life cycles, requirements for better quality and growing competition. The quality requirements are generally linked to increased customer require-ments, whereas speedy and efficient development processes are the answer to shortened product life cycles.

Answers to today’s competitive situation can be found in the field of lean thinking, as lean methodology’s core ideas are focusing on customer value and eliminating waste. The potential to avoid waste and increase efficiency arises even in the early stages of the product development process.

Value stream analysis is one useful methodology, where the flow of infor-mation during product development is modeled to identify waste and then optimized. While helpful, value stream analysis gives a retrospective, not predictive, view. Because of this, optimization measures only affect future projects, not current ones.

Since value stream analysis is time-consuming, it is only applied periodi-cally. Even more, efficiency improve-ments decrease over time as only a few of the identified measures can be implemented, and controlling them often is impossible. Another approach to optimizing development projects is failure mode and effect analysis (FMEA). While FMEA does look to the future, the analysis relies on experience as opposed to existing data.

With so much data available – data that previous generations could only dream of – a tool that extends approaches for value stream analysis by using data from previous projects to predict deviations in future development processes could be extremely helpful.

The deviation probability map fits this profile. It is a data-based risk analysis tool that looks to the future by combining value stream analysis with predictive analytics. Data collected from value stream analyses are analyzed with the aid of data mining and statistical approaches to anticipate time, cost

and quality deviations from the target process.

Deviations in time, cost and quality measure a development project’s ineffi-ciency, so this methodology can help by preventing such deviations. The deviation probability map identifies hot spots, where problems are likely to occur, much like the kernel density estimation used in predictive crime analytics. The derivation is explained based on the steps to formulate a hot spot map for predictive crime analytics. Then the idea of predictive crime analytics is trans-ferred onto product development.

This article describes how to put together the deviation probability map, including a case study that presents such a map generated from real data collected during a value stream analysis. Finally, we give an outlook and point out where further research is needed.

Partners in crimeThe methodology is based on the hot spot analysis used in predictive analytics. This theory maintains that where crimes happened in the past are the areas where crimes will happen in the future. Likewise, activities that included deviations in the past likely will include deviations in the future.

Predictive crime analytics uses maps to point out hot spots, or areas with high crime rates. Our methodology transfers data collected via a value stream into a visualization tool, comparable to the kernel density estimation used to predict hot spots for future criminal activity. For example, the left side of Figure 1 shows a kernel density estimation that visualizes the distribution of crimes in Baltimore. The right side of Figure 1, the deviation probability map, aims to identify areas or types of activities with a high probability of deviation. In this map, every pillar indicates a specific activity type, and the pillar’s height indicates the average deviation of that particular activity.

Comparable to neighborhoods, which are characterized by a similar infra-structure, the activities located closely to each other show comparable charac-

12 Industrial Management

teristics. This map can be drawn from value stream data and used to anticipate deviations in a future development project.

The approach to derive and apply the deviation probability map is divided into two phases. The first phase can be seen as a process of “knowledge discovery in databases,” or a KDD-process. The second phase represents the predictive analytics part, where the knowledge gained in the KDD-process is used to predict devia-tions in the future development project.

In phase one, relevant data is prepared for the application of statistical methods and data mining algorithms. Multidimensional scaling transforms the data into a two-dimensional activity map that visualizes the dissimilarity of the described activities. To include the third dimension, which is necessary to complete the deviation probability map, deviations are allocated to the different activity paths, as shown in Figure 1. The resulting deviation probability map then can be evaluated and interpreted.

The second phase describes how future activities can be integrated into the deviation probability map in order to rate the deviation probability.

Before the methodology is described, the relevant data for the analysis will be defined, following the KDD-process described in “From Data Mining To Knowledge Discovery in Databases” from AI Magazine. Examining the value stream of development projects can identify the following types of data:

• Activities can be seen as the core information within a value stream analysis, as they describe the processing of information in order to complete a product.

• Deviations, in this context, represent the quantification of inefficiencies. Therefore, they are also relevant for the data mining analysis.

• The last information found within a value stream are the influencing factors, which affect the execution of activities. These factors can be found within the value stream as interrela-

The activities of a development

project need to be

transformed into a map

before deviations can

be allocated to each activity.

tions of activities. Influencing factors also are found at the corporate level in terms of framework conditions for a project.

Building the deviation probability mapJust like in the field of predictive crime analytics, which uses street maps to allocate crimes to specific areas, the activities of a development project need to be transformed into a map before deviations can be allocated to each activity.

In a street map, neighborhoods can be described by similar characteristics, such as the condition of the houses or the surrounding infrastructure. Compa-rably, on our map activities need to be positioned near one another depending on their similarity. The positioning of activities in a two-dimensional plot is done by aid of multidimensional scaling. So the different activities first have to be described by consistent character-istics. Afterward, you can calculate the similarity of the different activities and apply the multidimensional scaling algorithm. Figure 2 summarizes the steps to derive the deviation probability map, which also are described below.

Step one: Classifying the activities. Attributes must be defined to describe all the activities that happen during a development project. The attributes can be related to either implementation or time, but both are important. The goal of the deviation probability map is to

identify hot spots on an implementation level, but attributes that relate to time are relevant because of how activities within a value stream are interrelated.

The “activity type” describes the value-adding share of an activity. Core activities directly create value. Support activities, on the other hand, provide resources for the core activities. For example, management activities strive to support the cooperation of the other activity types, therefore often inheriting organizing characteristics.

The “type of execution” helps transform an activity description, which is recorded in prose, into a standardized description. The characteristics that we use to define the type of execution are decision, analysis, realization, planning, information, steering, controlling and consultation. The “degree of standard-ization” divides activities in creative and repetitive activities, defining the planning capability of the outcome. The attribute “responsible department” allocates the activity to the executing department. The final attribute describes whether an activity is executed between divisions.

In total the attributes holistically describe how each activity is imple-mented, as they define the general type of the activity and answer the important questions of what is done by whom, how and in cooperation with whom. With these activity classifications in place, each activity can be described by aid of a 5-by-1 activity vector (see

MAPPING THE FUTUREFigure 1. The kernel density estimation on the left comes from the Rand Corp.’s Predictive Policing: The Role of Crime Forecasting in Law Enforcement Operations. It visualizes the distribution of crimes in Baltimore, while the deviation probability map on the right identifies development project activities with a high probability of deviation.

january/february 2016 13

step one in Figure 2) that contains the characteristic of each attribute in the respective row.

Step two: Determining and visualizing dissimilarities on the activity map. Based on the classifi-cation of activities, a binary distance vector is used to calculate the dissimi-larities, marked on our maps as distance, between the activities. To derive a binary distance vector, two activity vectors are compared row by row. If two character-istics of an attribute are the same within both activity vectors, the value of the binary distance vector in the respective row is zero. Otherwise the value is one (see Figure 2, step two).

The distance vector’s length describes the differences between the two respective activities. In order to visualize the dissimilarities of all activities in a two-dimensional plot, the distances between all activities are calculated and inserted into a distance matrix. To transform the calculated similarities of the activities into a two-dimensional plot, scaling is applied. This two-dimen-sional plot, what we call the activity map, is the equivalent of a street map.

Within the activity map, similar types of activities are closer to each other than different types of activities. Hence, the clusters within the diagram can be seen as activity neighborhoods.

Step three: Completing the deviation probability map. Allocating deviations to the different types of activities turns the activity map into the deviation probability map, comparable to the kernel density estimations used in predictive analytics.

The map’s main purpose is to identify and visualize the average deviation of different types of activities. To visualize these deviations, they have to be quantified. Since the focus of this research is to increase the efficiency of development projects, deviations are measured in the three dimensions common for process efficiency: time, cost and quality.

Every activity has a defined budget, time of completion and expected quality level. Whenever an activity does not

Deviations are measured

in the three dimensions

common for process efficiency:

time, cost and quality.

fulfill one of the defined parameters, a deviation must have occurred. So far, the magnitude of a deviation only can be recorded qualitatively by aid of a scale of values that consist of zero, three, six and nine. To quantify all the deviations found in the value stream data, they were rated retroactively based on their description. The cumulative score of all three dimensions summarizes the total deviation allocated to one specific activity, defining that activity’s degree of inefficiency.

After quantifying all identified devia-tions, the average deviation of all activity types has to be calculated and allocated to each respective activity path. To calculate the average deviation of one activity path, which is represented by one point within the activity map, the deviations of all activities described by this path have to be added up. This sum is then divided by all activities with a dissimilarity of zero, as shown in step three of Figure 2. By allocating the

average deviations of all activities with sij = 0 to the respective activity path, the two-dimensional activity map is transformed into the three-dimensional deviation probability map.

Now that the map has been finalized, the KDD-process is completed. Project managers and executives can see the hot spots on the map and the patterns involving where deviations occur.

Applying your map to future development projectsThis knowledge based on recorded data from past projects can be used to increase the efficiency of future development projects. Continu-ously improving and maintaining the deviation probability map can help to make more precise predictions and identify how deviations develop.

The deviation probability map can help a development project leader evaluate a future phase, such as the next stage of a gate-oriented development

FOLLOW THE PROCESSFigure 2. Three steps are involved in constructing deviation probability maps: (1) Classifying the activities; (2) determining and visualizing dissimilarities within the activity map; and (3) deriving the actual map.

14 Industrial Management

process after passing a quality gate. Within a gate-oriented development process, many companies know which activities will be executed in an upcoming stage. This knowledge comes from past development projects and the company’s standardized development process. This allows the project leader to define the upcoming activities. These activities already have been described during step one of the methodology, and multidimensional scaling can help insert any new activities into the deviation probability map.

When a development project requires new activities that aren’t on the map, there are two possibilities. First, the “new” activity path could already exist in other projects. If so, the expected deviation can be determined directly.

On the other hand, the new activity might be entirely new and not have any data associated with it. In this case, the analysts should associate the new activity path with a cluster of similar activity paths, using the cluster’s average deviation to define the new activity’s expected deviation.

Depending on the expected deviation in time, cost and quality, project teams can implement preventive measures. For example, if a deviation is expected during an activity that must be under-taken by multiple divisions, the project leader could demand that all relevant departments integrate their processes early. This could prevent the problem from cropping up, increasing the efficiency of the entire project.

When applying the methodology, it is important to continue adding more data to the database to show possible shifts of deviation hot spots. Furthermore, a growing database allows more precise predictions. So after executing activities and discovering deviations, always add them to the database.

Case studyTo evaluate the proposed methodology, nearly 300 activities were described by aid of the defined attributes and converted into the deviation probability map shown in Figure 3. The activities

The deviation

probability map ...

supports project leaders

by helping them analyze future devel­

opment stages and implement

optimi zation measures

before problems rear

their head.

were recorded during a value stream analysis of a development project for electromechanical components.

When the company did the project, it did not initially strive to measure deviations and quantify inefficiencies. So the research team quantified the data retroactively.

As can be seen on the map, it is intuitively possible to identify deviation hot spots. The upper area of Figure 3 shows a deviation hot spot that involves interdivisional activities, something managers need to target with interven-tions to prevent problems and execute the project. However, examining the bottom corner reveals few problems with deviations involved with the actual mechanical construction.

As a project’s next phase is coming up, managers and leaders can examine the planned activities and their expected deviations to come up with preventive measures. In this case, a series of meetings between divisions at the beginning of the product marketing phase could hash out the problems before they take too much time, cost too much money and degrade the quality of the product being developed.

In times of shortened product life cycles and rising quality require-ments, manufacturing companies must eliminate waste and deviations to increase the efficiency of their product development projects. Using a value stream analysis is a common approach to analyze development projects and define optimization measures retrospec-tively. But this retrospective view does not optimize a project that has already happened.

Introducing the deviation probability map as a tool supports project leaders by helping them analyze future devel-opment stages and implement optimi-zation measures before problems rear their head. And unlike other approaches that look to the future, these maps are based on data, not experience.

Future research could support the automatic quantification of deviations, perhaps supported by a product life cycle management system. Another focus of future work could identify deviations by aid of leading indicators. Such indicators could enable a more dynamic evaluation of a development project rather than the rather static hot spot analysis of the deviation probability map. v

PROBLEMS AT A GLANCEFigure 3. This deviation probability map shows that in this organization problems crop up in interdivisional activities that involve product marketing, but mechanical construction seems to have its act together.