Ecole Centrale Paris

Laboratoire Génie Industriel

Cahier d’Études et de Recherche / Research Report A MULTI-CRITERIA ASSISTANCE TO THE CHOICE OF RISK MANAGEMENT METHODS IN PROJECTS Franck MARLE et Thierry GIDEL CER 10– 24 Décembre 2010

1

A MULTI-CRITERIA ASSISTANCE TO THE CHOICE OF RISK MANAGEMENT METHODS IN PROJECTS

Franck MARLE et Thierry GIDEL (UTC)

Abstract: As projects are facing tight constraints, uncertainty and change, risk management is a very

important issue in project management. Our goal is to provide a project office manager or a project

manager with one or more adequate Project Risk Management (PRM) methods. In order to achieve

this goal, we propose a typology of PRM methods and a list of criteria that should be considered when

choosing the methods, by screening and ranking. Finally, we propose a Multi Criteria Decision

Making (MCDM) model that could be used to select the methods. An application on an industrial case

study is presented and some conclusions and perspectives are drawn.

Keywords: multi-criteria decision-making; fuzzy numbers; method choice; project management; risk

identification; risk analysis

2

1. Introduction

It is admitted today that Project Risk Management (PRM) has an important role to play on Project

Management and project success rate. This is particularly true in the development of a new product or

system, which is a wide process that includes both project and product lifecycle aspects. In this paper,

we focus on the choice of a relevant PRM method in new product/system development projects. Our

goal is to provide project office managers, project managers or any decision–maker (like risk

managers) with a framework and its associated tools to make the adequate choice. The new product

development project is extremely complex and involves the implementation, not only of product

development practices, but also of project management activities. Standards like ISO 10006

(International Standards Organization), PMI (Project Management Institute) or IPMA (International

Project Management Association) establish that project management consists in the planning,

organization, monitoring, control and reporting of all the aspects of a project, and in the motivation of

all people involved in reaching the project objectives (ISO 2003; PMI 2004; IPMA 2006). According

to PMBOK (PMI 2004) and AFNOR NFX50-117 standard (AFNOR 2003), project risk is defined as

“an uncertain event or condition that, if it occurs, has a positive or negative effect on at least one of the

project objectives”. If these risks are not managed in a pro-active way using a structured approach,

then they can result in serious consequences for the project, as said in ISO 10006 (ISO 2003). PMI

describes the PRM purpose as “the increase of probability and impact of positive events, and the

decrease of probability and impact of negative events” (PMI 2004). As a consequence, various risk

management methodologies have been developed : some standards have proposed risk management

methodologies, which are specific to project context or generic (IEC 1995; APM 1996; AFNOR 1997;

AFNOR 1999; IEEE 2001; BSI 2002; PMI 2004; IPMA 2006). They may have been introduced in

different fields, like project management, systems analysis, design, insurance, food industry,

information systems, chemical systems, industrial safety. Yet, when looking at companies practices,

once can observe that PRM methods are not so widely used (Coppendale 1995; Lyons and Skitmore

2004). When a method is implemented on a project, it is often either imposed by a corporate standard

3

or chosen by the project leader, because he already has tested it. Of course, leading companies and

organisations are implementing various PRM methods. For instance, according to (Tumer, Barrientos

et al. 2005) and (Kurtoglu and Tumer 2007), NASA implements various PRM methods like Failure

Modes and Effects Analysis, Fault Tree Analysis, Probabilistic Risk Analysis, etc. which will be

detailed in the following part 2.

The first issue addressed in this paper is that the PRM method used in a company may have been

chosen for wrong reasons or for historical reasons which are obsolete in the current context. The

second issue is that a given company may have several methods fulfilling the same needs, often for

historical reasons of local use of each method. This makes it difficult to combine these methods in an

efficient and effective way.

To tackle these issues, we define a Multi-Criteria Decision-Making (MCDM) process which will

evaluate, eliminate and rank PRM methods alternatives. An industrial case study is presented in order

to test a first implementation of the methodology. For defining a MCDM process, several questions

have to be addressed, like the goal (choosing, sorting, ranking, screening), the possible alternatives

(the PRM methods), the selected criteria, the mode of evaluation of each alternative among each

criterion, the mode of aggregation for evaluating alternatives using their local evaluations and the

mode of selection regarding the final evaluation and ranking of the alternatives. In this paper, we

chose to use a combination of screening and ranking, and a combination of qualitative and fuzzy

evaluations. These evaluations are aggregated using a geometric weighted average.

The paper is then organized as follows. The application field of Multi Criteria Decision-Making is

presented, with a focus on the use of fuzzy set theory as some judgements being subjective and not

precise enough (part 2). A literature review of PRM methods enables us to propose a typology of these

methods and a list of criteria to select them (part 3). Then, our PRM method choice process is

introduced (part 4). The decision-making process is detailed in three steps, some of them having a

variable depth degree depending on the ambition of the decision-maker and on the context of the

4

company. Finally, an industrial case study is presented (part 5) and some discussions and conclusions

are drawn (part 6).

2. Background on Multi-Criteria Decision-Making

The expertise required to choose a method may be very deep, because of the use of some advanced

concepts, like Markov, Monte-Carlo or logical gates, etc… Usually, only experts of the field are able

to take full advantage of a PRM method. The issue of PRM methods choice (RIM/RAM) becomes

even more important when it is applied on a decentralized way, sometimes with local or web-based

software, without any technical support, from a project office for instance, or a risk manager or an

expert. Both the characteristics of PRM methodologies and preferences of the decision-maker are to be

modelled. The goal is to use a Multi Criteria Decision Making (MCDM) model that enables to match

the decision maker’s preferences. Preferences are often uncertain and expressed by linguistic terms,

such as « good », « very much », « I prefer », which may require the use of fuzzy set theory in addition

of the use of a MCDM method. Classical MCDM methods suppose to conduct an evaluation of some

alternatives regarding some criteria, by using qualitative or quantitative scales, crisp or fuzzy values,

and direct or comparison-based evaluations. Our goal in this paper is to propose one or more adequate

PRM methods to the decision-maker, which supposes to combine screening and ranking. Some

evaluations are based on facts and are quantitative, and some are based on human judgement and may

be qualitative or even sometimes fuzzy. This paragraph introduces general MCDM problems and

methods. It consists in describing the possible objectives, the classical steps of decision-making

process, and the classical principles and approaches. We will explain why we decided to use in our

research method a combination of screening and ranking, and a combination of qualitative and fuzzy

evaluations.

2.1. Possible objectives for a MCDM problem

Three main issues exist in MCDM:

5

• Choice: choose the best alternative. For a basic decision-making problem of choosing one or

several best alternatives, it is useful to begin by eliminating those alternatives that do not

appear to warrant further attention, which is called screening (Hobbs and Meier 2000).

Screening is the process that reduces a set of alternatives to a smaller set of alternatives that

(most likely) contains the best one. It supposes to have some elimination thresholds, or

intervals, on evaluation scales.

• Ranking: rank all alternatives from best to worst. It supposes to have a global evaluation

model for each alternative, taking into account all the considered criteria and their different

scales.

• Sorting: sort all alternatives into different pre-ordered groups

In this work, we consider only screening and ranking categories.

2.2. Classical steps for a MCDM process

A global MCDM problem follows the serial process of:

• defining decision objectives,

• identifying and arranging criteria, with potential interdependencies,

• identifying and arranging alternatives, with potential interdependencies,

• evaluating criteria , with weights and thresholds,

• evaluating alternatives for each criteria and with a global model,

• screening out alternatives which do not fit to criteria thresholds

• ranking remaining alternatives according to their individual evaluations and criteria weights

• making the decision.

For a set of criteria C={Ci}, i=1 to NC (number of criteria) and a set of alternatives A={Aj},j=1 to NA

(number of alternatives), the evaluation of Aj regarding Ci is called Eij.

6

7

2.3. Classical principles for the decision-maker’s preference expression

Two kinds of evaluations do exist: the values (preferences on consequences) and the weights

(preferences on criteria). Basically, they can be evaluated whether directly or indirectly via relative

comparison (often pair wise comparison). Moreover, they can be quantitative, or qualitative, or even

fuzzy when the degree of precision and reliability is not enough to get crisp qualitative evaluations.

For instance, linguistic decision-making uses linguistic expressions on criteria as constraints, and is

mainly used for screening. We distinguish lexicographic constraints and disjunctive/conjunctive

constraints. In the first case, criteria are ranked in order of relative importance, and all alternatives are

examined to assess whether the first criterion is satisfied. For those alternatives which are not screened

out, the process goes on to the second criterion, and so on until the last. Disjunctive or conjunctive

constraints express conditions involving more than one criterion, and are characterized by the use of

« and » and « or » operators.

For the values, the most known and used models are utility theory (KEENEY and RAIFFA 1976) and

outranking (Vincke 1992). It consists in a transformation of raw consequence information into

preference information which is useful for the decision-maker. Pareto optimality is a well-known

concept introduced by the famous scientist (Pareto 1971) which takes into account multiple criteria for

overall optimality. It is based on the domination concept, and is also called efficiency of an alternative.

A1 dominates A2, denoted A1 > A2, iff ∀ �??1.. ??, 1 ≥ 2 , with at least one strict inequality. Following this definition, an efficient alternative, also called Pareto

optimal alternative, is a non-dominated alternative:

8

A1 is a Pareto optimal alternative iff ∄ � > 1. For the criteria, it refers to expression of the relative importance of criteria. Usually, for each criterion

Ci, the associated weight Wi is strictly positive, and:

Two kinds of weights, trade-off weights (TW) and non trade-off weights (NTW) are defined in

(Belton and Stewart 2002). For TW, preferences are compared as they are aggregated into a single

expression, a phenomenon called compensation. Trade-off weights are essentially used for classical

aggregation models, like additive or average-based methods. It enables to study some phenomena like

sensitivity analysis (Rios Insua 1990), dominance and potential optimality (Hazen 1986;

Athanassopoulos and Podinovski 1997), which consist of changing inputs (values or weights) to look

at consequences on outputs. Non Trade-off weights, also called outranking, was introduced in (Roy

1996). An outranking relation is a binary relation S defined on A, with the interpretation that ASB if

there are enough arguments to decide that A is at least as good as B, while there is no essential reason

to refute that statement (Vincke 1992).

In this paper, we do not consider outranking methodologies but our evaluation will be done according

to single synthesis criteria principle, using multi-criteria aggregation and Pareto domination principle.

We chose to use a single weighted geometric mean. This is simple enough to avoid problems due to

methodology pre-requisites for our users. The choice of a geometric weighted average is made to

screen out methods which do not correspond to minimum thresholds. Finally, it permits to give a

better global assessment to balanced solutions, which is not the case with arithmetic mean.

For instance, (0,5*0,5)½ = 0,5 > (0,9*0,1)½ = 0,3, whereas (0,5+0,5)/2=(0,9+0,1)/2=0,5

2.4. Fuzzy set theory

9

Most of the preferences of the decision-makers are expressed by using linguistic expression, such as

« very likely », « highly preferable », « do not like ». They involve the use of fuzzy set theory, first

introduced by (Zadeh 1965; Zadeh 1975), and applied to decision-making by (Bellman and Zadeh

1970). Some definitions are given for usage in this paper, extracted from several references, like

(Kaufmann and Gupta 1991; Zimmermann 1991; Klir and Yuan 1995).

Definition 1 : a fuzzy set ñ in a universe of discourse X is characterized by a membership function

μñ(x), which associates with each element x in X a real number in the interval [0,1]. The function

value μñ(x) is termed the grade of membership of x in ñ (Kaufmann and Gupta 1991).

Definition 2 : a fuzzy set ñ is convex iff (Klir and Yuan 1995):

for all (x1, x2) in X and λ in [0,1]

Definition 3 : a fuzzy set ñ is normalized if its height is equal to 1 (Klir and Yuan 1995). The height is

the largest membership grade attained for any x in X.

Definition 4 : a fuzzy number is a fuzzy set that is both convex and normal (Kaufmann and Gupta

1991).

Definition 5 : a positive triangular fuzzy number ñ can be defined as (n1, n2, n3), with the following

membership function μñ(x) :

The fuzzy number is symmetrical iff n2=(n1+n3)/2.

10

Definition 6 : a linguistic variable is a variable whose values are linguistic terms, like very low, low,

medium, high and very high. Fuzzy numbers can represent these linguistic values in situations where

quantitative expressions or reliable qualitative expressions are not possible. In this paper, positive

triangular fuzzy numbers are used to express linguistic variables.

Definition 7 : in the case where several decision-makers give their opinion in terms of fuzzy numbers

ñk, k=1 to NDM (Number of Decision-Makers), then the aggregated fuzzy number ñ can be defined as

explained in (Amiri, Zandieh et al. 2009) :

Some methodologies have been developed thanks to the use of fuzzy set theory, like fuzzy Standard

Additive Weighting model, fuzzy weighted product model, fuzzy AHP, revised fuzzy AHP and fuzzy

TOPSIS, studied and compared in (Triantaphyllou and Lin 1996). A combination of principles of AHP

and fuzzy set theory gives the fuzzy AHP method, which has been applied by (Shamsuzzaman, Sharif

Ullah et al. 2003). In our case, we are going to use fuzzy weights, for expression of preferences of the

decision-maker on the criteria importance.

3. Background on Project Risk Management methods and proposal of a choice criteria list

We identify the main PRM methods and their main characteristics according to a literature review

focused on a local analysis (a specific method, a specific company, a specific journal or conference)

and on a global analysis (research by keywords, by application fields and knowledge areas). A list of

criteria that will be used for the choice is introduced as a synthesis of this part.

3.1. Description of the methods

The Project Management Institute (PMI 2004) presents project management in nine knowledge areas:

integration, scope, time, cost, quality, human resources, communications, procurement and risk

management. Risk management consists in the treatment of the project uncertainties through a

structured, four steps generic approach (PMI 2004):

11

1. Risk identification: describes the identifiable risks, that is to say the potential events that could

occur and lead to negative or positive impact on the project,

2. Risk analysis: analyzes causes and consequences of the identified risks, in order to evaluate

their criticality, mainly by assessing probability and impact.

3. Risk treatment (or response planning): decides tasks, budgets and responsibilities in order to

avoid, mitigate or transfer some risks, often the most critical in priority.

4. Risk monitoring and control: follow-up, by the identified responsible persons, of the planned

actions and of their impact on the criticality of the risks.

As detailed in (Marle and Gidel 2010), we propose to present the methods according to a typology

based on the nature of the identification: analogical, heuristic and analytical (Table 1).

Table 1: classification of PRM methods according to three types of risk identification approach

Finally, we have considered 32 RIM and 19 RAM, briefly described below in tables 2 and 3.

Particularly, table 3 indicates to which type each RIM belongs. A method may be used for more than

types, meaning that it can be used with or without experience.

12

Table 2: list of Risk Identification Methods (RIM) and their classification according to

experience/expertise

13

Table 3: list of Risk Analysis Methods (RAM)

14

3.2. Building a list of criteria

As there are plenty of PRM methods, it is hard for a decision-maker who is involved at the project

level or at the multi-project level, to select the most suitable methods for his particular context. We

propose a three level decomposition in order to identify the criteria that could be used to choose a

PRM method: the organization level, the project level and the decision-maker level. The description of

each criterion is detailed in (Marle and Gidel 2010). We then propose the following criteria for

selecting PRM methods, as described in Table 4.

The first four criteria (Organization level) are mandatory, because it would be risky for the company to

implement a method if it is not mature enough. They are measured for a company and are compared to

a type of methods (analogical, heuristics, analytic). The last criteria (Project and Decision-maker

levels) are preferences, as the company prefers a method which corresponds to its specific needs. Of

course, as these criteria are deducted from a literature review, they could be discussed, improved and

further tested to confirm their pertinence.

15

Table 4: description of choice criteria with their evaluation scale

16

Finally, the table 5 shows the evaluation of RIM and RAM among considered criteria.

Table 5: evaluation of each RIM and RAM according to considered choice criteria

17

4. Development of a multi-criteria decision-making process for Project Risk Management

method choice

The different PRM methods introduced in previous section are potential candidates for the company.

The choice is in three steps, which consist in gathering and treating data about methods, criteria and

company (see figure 1):

• A selection of the RIM which correspond to company maturity,

• A selection and ranking of the RIM/RAM methods among the company preferences,

• A final choice of a combination {RIM+RAM} of methods depending on their ranking, their

links and their implementation effort.

The output of these three steps is the decision, followed by the implementation of the choice.

1

2

3

4

DATA ABOUT THE METHODS AND

CRITERIA

DATA ABOUT THE COMPANY

DATA TREATMENT IN DECISION-MAKING PROCESS

1

2

3

IMPLEMENTATIONDECISIONStep 1Screen out

Step 2Screen out and rank

Company maturity Company preferences Company maturity

Assessment of methodsregarding criteria

Implementation issue

Compatibility

Step 3

RiskIdentification Methods

RiskAnalysisMethods

Figure 1: description of the decision-making process by screening and ranking

18

4.1. Step 1 : first selection of RIM regarding company maturity

Four criteria are introduced to define what company maturity in project risk management is. They are

called Company Maturity Criteria (CMCj), j=1 to 4. For each of these criteria, it is asked to the

decision-maker to assess his company, in order to get evaluations called CMj for Company Maturity

for the jth criteria. As seen in part 2, three types of RIM do exist: analogical, heuristic and analytic.

They are called Tk, k=1 to 3. For each type, a minimal maturity is mandatory in order to be able to

implement correctly a method of this type. We then define a threshold for each type Tk and each

criterion CMCj, called MTjk (for Maturity Threshold). We have the following constraint:

For each k [1..3], if there is j [1..4] such as CMj < MTjk, then the type Tk is not adequate for the

company and is screened out.

Then, we evaluate whether the RIMi belong or not to each type Tk. This binary matrix presented in

part 2 enables us to know whether each RIMi can be applied or not regarding company maturity. There

are two possibilities:

• RIMi belongs to only one Tk : if Tk is screened out, then RIMi is screened out

• RIMi belongs to more than one Tk (2 or 3) : if one Tk is not screened out, then RIMi is not

screened out. That means, that even if RIMi cannot be applied in a certain way (analytic for

instance), it can be applied in another way (analogical for instance).

From the initial list of RIM, this step gives a shorter list of potential candidates for next step.

4.2. Step 2 : selection and ranking of RIM/RAM regarding company preferences

This step is detailed for RIM; the same principles are applied for RAM. Only the terms and the values

change. It can be expressed as step 2a and step 2b, where step 2a is detailed below for RIM and step

2b is identical for RAM. The goal of this step 2a is to study the fitness of remaining RIM alternatives

to company’s preferences. These preferences are expressed among some criteria, called Company

Preferences Criteria CPCj. These criteria are expressed with a weight and a minimal threshold,

19

respectively called RIwj and RImtj. We then have to evaluate each RIMi among each CPCj, called Eij

and give a global evaluation Ei (Equation 1):

(1)

With the constraint: RIMi is screened out (RIMi=0) iff there exists j such as Eij< RImtj

We reformulate the constraint as following (Equation 2):

Eij [1..5] except if Eij<RImtj where Eij=0 (2)

In the case where NDM decision-makers give their opinion, then we have NDM expressions for each

threshold, called MTik, k =1 to NDM and the final threshold MTi is obtained as the maximum of the

individual thresholds. This enables to satisfy all the decision-makers, even if it is harder in terms of

selection.

The particularity of evaluation of company’s preferences is the use of fuzzy weights. They are

expressed on a fuzzy scale which transforms linguistic preference judgements into numerical intervals

on a scale [1..10]. The Figure 2 shows this scale.

0

0,2

0,4

0,6

0,8

1

1,2

1 2 3 4 5 6 7 8 9 10

N

VL

L

A

H

VH

Figure 2: description of the fuzzy scale used in our approach

20

It is to be noticed that some levels are particular:

• The level Negligible is not exactly triangular as n1=n2=1.

• The level High is larger than the others. It enables to make a bigger difference in scores when

preference is High or Very High. The goal is to emphasize the most important criteria for the

decision-maker, and try some avoid compensation phenomena.

As we have fuzzy weights, we obtain a fuzzy score by calculation of three scores for each

corresponding weight. A defuzzification formula is applied in order to obtain a final global score for

each method. The methods are ranked according to their global score. A multi-criteria Pareto

optimality analysis is done. The top 5 of the non-dominated methods is presented to the decision-

maker with individual and global scores. This step permits to screen out RIM which do not correspond

to company requirements, then to rank the remaining RIM in order to make the final choice described

in step 3.

4.3. Step 3: the final choice of a combination of (RIM+RAM)

This step may be done very quickly by choosing the first method in each ranking. We give hereafter

three ideas for refining this brutal choice:

• The significance of the gaps in the ranking

The reliability of the outputs depends on the reliability of the inputs. The mistakes and imprecision are

amplified or eliminated by the aggregated calculation. So, we argue that there should be a large

enough gap between two solutions in order to decide to choose one and to reject the other one. This is

why we used geometric aggregation and fuzzy weights. If maximal fuzzy score of solution A is

inferior to minimal fuzzy score of solution B, then we can be more confident on the choice of B and

the elimination of A. The decision-maker can decide at the beginning of the process of a minimal gap

MG between two solutions (Equation 3):

If E(RIMi1) > E(RIMi2)+MG, then the choice of RIMi1 is considered as reliable enough. (3)

21

If some solutions remain very close for instance for two RIM, a difference can be made with the

compatibility parameter described below.

• The choice of a combination of (RIM+RAM) instead of two independent choices

We introduce CR as the Compatibility Ratio between two RIM and RAM. We define the score of a

couple of RIM and RAM as following (Equation 4):

E(RIMj0, RAMj1)=E(RIMj0)*E(RAMj1)*CR(RIMj0, RAMj1) (4)

CR is equal to 1 when methods are independent or neutral, and is superior to 1 if methods do fit easily.

For instance, FMEA can be used both as a RIM or a RAM, then its CR is good. Brainstorming is a

standard RIM that does not have positive or negative influence on the use of RAM, so the CR is equal

to 1. On the contrary, the identification of risks thanks to cause trees is not adequate to the analysis of

the global risk exposure. These are different ways of thinking and different ways of using data, some

CR is inferior to 1. If the methods remain very close with the two first refinements, a third one is

possible by considering the change that the organization will have to do to implement each method.

• The organizational cost of implementation of a method can be a parameter of the final choice

We introduce IE as the Implementation Effort (see list of criteria) of a RIM/RAM in the company.

This ratio depends on both the method and the company. It is not a ratio which is independent of the

company. Then, we can apply a penalty/bonus to the initial score of a RIM/RAM by multiplying by

this index (Equation 5):

E’(RIMj0)=E(RIMj0)*IE(RIMj0) (5)

22

5. Industrial case study

An application of the choice method has been done on a company which delivers tramway

infrastructure for cities. The company was historically on product development and had recently

extended its scope by delivering to a city the product and its environment, that is to say the civil

engineering, the signalling material, the maintenance and storage depots, etc. As this type of project is

new for the company, the question of Project Risk Management method is pertinent, as risk

management for product development project is not the same for other areas.

The first action consisted in interviewing a person accountable for PRM method choice. He was a

decision-maker involved in several running projects, not a decision-maker from a project office. There

was in this case study no question about standardization of the method to the whole projects of this

type. The goal was only to test which RIM and RAM could best fit to these particular five projects.

The smallest project was about 5 years and 200 M€. This interview gave us information about

company maturity and company preferences, respectively CMj, RIwj, RImtj, RAwj and RAmtj. We

obtained the following results.

5.1. Step 1: first selection of RIM regarding the company maturity

Due to the team maturity level in risk management and due to the innovative level of the product, it

was difficult for the company to implement analytic methods (see Table 6).

Table 6: adequation of each type of PRM method to the company’s maturity

RIM which were only analytic were therefore screened out (RIM 16 and RIM 21).

23

5.2. Step 2 : selection and ranking of RIM/RAM regarding company preferences

The decision-maker expressed preferences in terms of weights with a linguistic scale [Negligible, Very

Low, Low, Average, High, Very high]. These linguistic variables were transformed in fuzzy numbers

with the intervals described in Figure 2. He expressed also minimum thresholds, the value 1 meaning

that there was no minimum threshold. The three criteria with a minimum threshold of 1 were not

significant, neither for screening out or for ranking as we used a geometric weighted product for

scoring methods. The other criteria did have an importance as they helped to reduce the number of

possible alternatives. The Table 7 below shows the expression of company’s preferences and methods

evaluation, and the scoring of the methods with their ranking.

This step enabled to screen out many methods. On the final ranking, only 6 RIM and 2 RAM were still

candidates.

24

Table 7: synthesis of calculations and results for the case study

25

5.3. Step 3 : the final choice of a combination of (RI+RA) methods

Three additional parameters may be included in order to refine the choice :

• The significance of the gaps in the ranking

In this case, 8 criteria were significant in the differentiation of methods, 2 for RIM and 6 for RAM.

This means that the minimal gap should be different for RIM and RAM. We obtain from the previous

table the fuzzy intervals for logarithmic scores. No obvious elimination can be done at this stage.

• The choice of a combination of (RIM+RAM) methods instead of two independent choices

The final combination in this case was a choice between local and global cause-effect analysis.

Namely, both methods were of the same category (tree-based or cause-effect methods). The difference

was on the scope of the analysis. For instance, root-cause analysis (RIM 17) is a deeper cause-effect

analysis looking for several cause-effect relationships. Chain reactions and depth analysis of causes

are the focus. In more local methods, like FMEA (RIM 6), the causes and effects are analyzed at only

one level, which means direct causes and direct effects.

As the decision-maker decided to proceed to deeper analysis, the choice of Ishikawa was made for

RIM (RIM 20 with experience) and the compatibility was good with RAM (RAM 5 with experience).

He decided to implement also RIM 17 on specific past problems, and to locally implement RAM 3

where there was some novelty on the current projects. Finally, he made a mix between global

analogical methods based on experience and local heuristic methods based on expertise, on some

points only. This is possible as the whole methods are based on the cause-effect modelling principle,

which makes them very compatible.

• The organizational cost of implementation of a method can be a parameter of the final choice

All the candidates are easy to understand and to use. Only FMEA could be judged as more difficult

than Ishikawa and root-cause analysis, but it conducts to the same choice. In this case, the

organizational cost of implementation does not change the final choice, it is not necessary to make a

trade-off.

26

5.4. Result of the case study

The growing complexity of projects involves a huge number of cause-effect relationships between

parameters, resources and events. These relationships can cause surprises like reaction chains or loops,

which are quite impossible to detect in the current situation. The current PRM method implemented in

the company is mainly based on independent analysis and treatment of risks. The decision-maker

noticed the gap between his preferences, which involve the choice of interactions-based methods, and

the current implemented method. He was truly confident that the proposed ranking corresponds to his

inputs and to the company’s needs (for this type of projects). Particularly, the fact that the current

method is eliminated in our approach has been a surprise for him, but it has been well accepted. He

was also surprised by the huge number of methods which were eliminated with his requirements of

minimum thresholds.

Even if the decision-maker agreed with the methodological recommendation, he predicted a tough

change from the current situation to the desired one. Namely, the methods in themselves are not

difficult to implement, but the team maturity in risk management is very low. This can involve

difficulties to understand the benefit of including cause-effect relationships between risks inside the

global process.

The decision-making process took 1 hour to introduce the approach and the criteria, and to assess

company’s maturity and preferences. Then, the scoring of the methods is done instantaneously and the

results can be analyzed immediately.

6. Conclusions

6.1. Synthesis of the approach

This study points out that there are plenty of methods for Risk Identification and Risk Analysis, and no

guidelines to help users and decision-makers to select the most suitable ones for their particular

projects. In this paper, we propose an approach to help choosing the right project risk management

method considering maturity level, innovation degree, effort needed to implement the method, etc. Of

27

course, this could help project manager or project office manager in their decision making process.

The proposal is a three-step decision-making process, where the first step screens out methods which

are too ambitious compared to the company’s maturity, the second one screens out and ranks

remaining methods according to company’s preferences, and the third one refines this choice thanks to

some additional parameters, like gap analysis, compatibility between methods and organizational cost.

As there are several screening phases, we used a geometric weighted product for global scoring of

alternatives. As some judgements are given in linguistic terms, fuzzy numbers are applied in this study

to determinate the weights.

6.2. Discussion and perspectives for future work

Some points can be discussed about the sensitivity and robustness of the final result:

• About the methods evaluation:

Different choices have been made, like threshold definition, discrete scales for evaluations and

classification of methods into three types. It is obvious that final results are sensitive to these inputs,

but :

o The thresholds have been assessed by interviews and literature review,

o Lots of methods are classified into more than one type, which reduces the risk to

screen out a method abusively if a type is eliminated,

o It is necessary to use qualitative scales as no obvious quantitative parameters do exist,

• About the Decision-Maker’s (DM) preferences and evaluation of the company:

As we wanted to test those criteria, we developed a simple Multi Criteria Decision Making (MCDM)

model that could be used to select the methods and apply it to an industrial case study. This is a first

step toward a more robust model, but at this stage, it raises some questions about DM: what is the

sensitivity of this model to DM unreliability? Who should use it and when? Should it be used at

corporate level (project office) or by the project manager before starting its project?

28

In the case where NDM DM give their opinion, then we have NDM expressions for each parameter. The

final parameter is obtained by combination of the individual ones. In the case of crisp numbers

(company maturity assessment or company preference thresholds), the global number is the minimum

of individual values. In the case of fuzzy numbers (company preference weights), it follows the

principles given previously in definition 7 of section 2.2.

A complementary work is also ongoing on the use of fuzzy AHP for weighting the preferences. This

will give a more precise evaluation of the weights, since criteria will be pair wise compared.

• About the completeness of the PRM lists:

The list of methods is quite representative of what exists in literature and what is really applied in

companies. We can be confident that no important method is forgotten. For the list of criteria, it is

different as it is our own creation. But the validity of this list has been tested by studying and

analysing the characteristics of the methods in the literature; this could become choice criteria. Are

there other relevant criteria to take into account when choosing PRM methods? Is there any need for

new methods or for a combination of existing methods? Would it be interesting to define a standard to

evaluate PRM methods and notably their conditions of application and validity domain?

6.3. Added value of our approach

Finally, we argue that this decision-making process has an added value for the Project Risk

Management process of the company, and then for the Project Management process. Namely, a more

suitable RIM/RAM will enable to reduce the impact of the risks and to reduce the probability for these

risks to occur. So, both the rate of success and performance level of projects could be improved. A

perspective of development could be to use this decision-making process as a functional requirement

definition of a good RIM/RAM, and then to develop a specific and more suitable method. This method

could be built by compilation of existing methods (the most frequent) or by specific development.

Finally, we think that, classifying the existing methods might help to identify a lack in some aspects of

these methods. For instance, we find very few project management methods that can handle properly

the interaction between risks. Finally, we think that this study will permit managers to be aware of all

29

available PRM methods and that it will lead them to consider the choice of the method as a strategic

decision that could impact the project success.

7. References

AFNOR (1997). ISO 10006. Management de la qualité: lignes directrices pour la qualité en management de projet, Editions AFNOR.

AFNOR (1999). RG AERO 000 40, dans NF X50 410. Recommandation générale pour la spécification du management de programme, Editions AFNOR.

AFNOR (2003). FD X 50-117: Management de projet, Gestion du risque, Management des risques d’un projet, AFNOR.

Amiri, M., M. Zandieh, et al. (2009). "A hybrid multi-criteria decision-making model for firms competence evaluation." Expert Systems with Applications: doi:10.1016/j.eswa.2009.04.045.

APM (1996). Project Risk Analysis & Management (PRAM) Guide. High Wycombe, ASSOCIATION FOR PROJECT MANAGEMENT.

Athanassopoulos, A. and V. Podinovski (1997). "Dominance and potential optimality in multiple criteria decision analysis with imprecise information." Journal of the Operational Research Society 48: 142-150.

Bellman, R. and L. Zadeh (1970). "Decision-making in a fuzzy environment." Management Science 17: 141-164.

Belton, V. and T. Stewart (2002). Multiple Criteria Decision Analysis: an integrated approach. Dordrecht, Kluwer.

BSI (2002). ISO/IEC Guide 73:2002. Risk Management – Vocabulary – Guidelines for use in standards. London, BRITISH STANDARD INSTITUTE.

Coppendale, J. (1995). "Manage risk in product and process development and avoid unpleasant surprises." Engineering Management Journal.

Hazen, G. (1986). "Partial information, dominance and potential optimality in multi-attribute utility theory." Operations Research 34: 296-310.

Hobbs, B. and P. Meier (2000). Energy Decisions and the Environment: a guide to the use of multicriteria methods. Massachusetts, KLUWER.

IEC (1995). CEI/IEC 300-3-9:1995 Risk Management: part 3 – guide to risk analysis of technological systems. Geneva, INTERNATIONAL ELECTROTECHNICAL COMMISSION.

IEEE (2001). IEEE Standard 1540-2001: standard for software life cycle processes – risk management. New York, INSTITUTE OF ELECTRICAL AND ELECTRONIC ENGINEERS.

IPMA (2006). IPMA Competence Baseline (ICB), Version 3.0, March 2006 International Project Management Association.

ISO (2003). ISO 10006 - Quality Management Systems - Guidelines for quality management in projects. Switzerland, International Organization for Standardization.

Kaufmann, A. and M. Gupta (1991). Introduction to fuzzy arithmetic: theory and application. New York, Van Nostrand Reinhold.

Klir, G. and B. Yuan (1995). Fuzzy sets and fuzzy logic: theory and applications. USA, Prentice Hall Inc. Kurtoglu, T. and I. Tumer (2007). A graph based framework for early assessment of functional failures

in complex systems. DETC 2007., Las Vegas, . Lyons, T. and M. Skitmore (2004). "Project risk management in the Queensland engineering

construction industry: a survey." International Journal of Project Management 22(1): 51- 61. Marle, F. and T. Gidel (2010). Analysis of existing Project Risk Management methods, Ecole Centrale

Paris / Université Technologique de Compiègne.

30

Pareto, V. (1971). Manual of Political Economy (English translation). New York, AM Kelley Publishers. PMI, S. C. (2004). A guide to the project management body of knowledge (PMBOK) (2004 ed.).

Newton Square, PA, USA. , Project Management Institute. Rios Insua, D. (1990). Sensitivity Analysis in Multiple Objective Decision Making. Berlin, Springer-

Verlag. Roy, B. (1996). Multicriteria methodology for decision aiding. Dordrecht, Kluwer. Shamsuzzaman, M., A. Sharif Ullah, et al. (2003). "Applying linguistic criteria in FMS selection: Fuzzy-

set AHP approach." Integrated Manufacturing Systems 14(3): 247-254. Triantaphyllou, E. and C. Lin (1996). "Development and evaluation of five fuzzy multiattribute

decision-making methods." International Journal of Approximate Reasoning 14: 281-310. Tumer, I., F. Barrientos, et al. (2005). Towards risk based design (RBD) of space exploration missions:

A review of RBD practice and research trends at NASA", , . ASME/DETC 2005, Long Beach, California.

Vincke, P. (1992). Multi-criteria decision-aid. New York, Wiley. Zadeh, L. (1965). "Fuzzy sets." Information and Control 8: 338-353. Zadeh, L. (1975). "The concept of a linguistic variable and its application to approximate reasoning."

Information Sciences 8(1): 199-249. Zimmermann, H. (1991). Fuzzy set theory - And its applications Springer.