master thesis - it4bi · master thesis maximiliano ariel lópez leveraging decision aiding and...

Decision Support and BusinessIntelligence

Information Technologies for Business Intelligence

Master Thesis

Maximiliano Ariel López

Leveraging Decision Aidingand Bringing it Closer

to All Citizens

prepared at Idées du Sud SASUDefended on September 3–4, 2015

Advisor : Valentina Ferretti - Politecnico di Torino [email protected] : Nacéra Bennacer - CentraleSupélec [email protected]

Acknowledgments

Many people have contributed to the achievements of this challenging and estimu-lating Thesis Project. I would not have attained anything without their invaluable help.

To start with, I would like to thank my family for their unequivocal, omnipresentand everlasting support.

My gratitude also goes to the IT4BI Teaching and Coordination Teams formaking this amazing experience possible and for granting me the honour of being part of it.

Furthermore, I would like to acknowledge Prof. Valentina Ferretti, a great per-son and professional, for her advice, dedication and patience over the course of this project.

I would also like to thank Prof. Nacéra Bennacer and École Centrale Paris forallowing me the necessary freedom to undertake this unconventional initiative.

Last but not least, I would like to express my gratitude towards all the people whohave shared their time to test ElectioVis and provide their feedback to help me continueimproving it.

Contents

1 Introduction 11.1 A Foreword about Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Main Complexities and Challenges of Decisions . . . . . . . . . . . . . . . 2

1.2.1 Irreversibility and Opportunity Cost . . . . . . . . . . . . . . . . . 21.2.2 Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.3 Multiple Conflicting Objectives . . . . . . . . . . . . . . . . . . . . 21.2.4 Multiple Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.5 Far-reaching Consequences and Environment Dynamism . . . . . . 3

1.3 General Goals and Structure of the Document . . . . . . . . . . . . . . . . 41.4 Elements of Innovation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Multi-Criteria Decision Analysis Methods 72.1 Decision Aiding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Multi-Criteria Decision Aiding (MCDA) . . . . . . . . . . . . . . . . . . . 72.3 The Need for Choosing Among Multiple MCDA Methods . . . . . . . . . 82.4 A High-Level Overview of MCDA Methods . . . . . . . . . . . . . . . . . 8

2.4.1 School-Based Classification . . . . . . . . . . . . . . . . . . . . . . 82.4.2 Approach-Based Classification . . . . . . . . . . . . . . . . . . . . . 10

2.5 Methods Implemented during the Initial Phase . . . . . . . . . . . . . . . 102.5.1 Analytic Hierarchy Process (AHP) . . . . . . . . . . . . . . . . . . 112.5.2 ELECTRE III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.5.3 Multi-Attribute Value Theory (MAVT) . . . . . . . . . . . . . . . 12

2.6 Other Works on MCDA Method Choice . . . . . . . . . . . . . . . . . . . 132.7 Introducing the “Parliamentary” Approach . . . . . . . . . . . . . . . . . . 16

2.7.1 Non-hierarchical Nature . . . . . . . . . . . . . . . . . . . . . . . . 162.7.2 Reducing Ambiguity and Complexity . . . . . . . . . . . . . . . . . 172.7.3 Prioritising Questions . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.8 How Selection Works in the “Parliamentary” Approach . . . . . . . . . . . 17

3 A Survey on Current MCDA Software Solutions 213.1 Non-Commercial Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.1.1 diviz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.1.2 GMAA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.1.3 iMOLPe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.1.4 jMAF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.1.5 jRank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.1.6 JSMAA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.1.7 MCDA-ULaval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.1.8 WWW-NIMBUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.2 Commercial Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.2.1 1000Minds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

iv Contents

3.2.2 D-Sight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2.3 Expert Choice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.2.4 Hiview3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.2.5 OnBalance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.2.6 Smart Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.2.7 SuperDecisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.2.8 V.I.S.A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.2.9 Visual PROMETHEE . . . . . . . . . . . . . . . . . . . . . . . . . 37

4 ElectioVis: A New MCDA Application 394.1 Remarkable Features of ElectioVis . . . . . . . . . . . . . . . . . . . . . . 39

4.1.1 MCDA Method Choice . . . . . . . . . . . . . . . . . . . . . . . . . 394.1.2 Value Functions (MAVT) . . . . . . . . . . . . . . . . . . . . . . . 414.1.3 Swing Weights Elicitation (MAVT) . . . . . . . . . . . . . . . . . . 454.1.4 Revised White Cards Approach (ELECTRE III) . . . . . . . . . . 464.1.5 Monte Carlo Methods (AHP and MAVT) . . . . . . . . . . . . . . 474.1.6 Contextual Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5 Entrepreneurship, Legal and Administrative Aspects 495.1 Logo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495.2 Company Registration in France . . . . . . . . . . . . . . . . . . . . . . . 50

5.2.1 Public Registry of Commerce and Companies . . . . . . . . . . . . 505.2.2 Revenue Collection Service . . . . . . . . . . . . . . . . . . . . . . 51

5.3 Brand Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.4 Database Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

6 Usability Testing and Feedback Received 536.1 Registered Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536.2 Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.3 Method Choice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566.4 Survey Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596.5 Survey Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

6.5.1 Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626.5.2 User Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636.5.3 On-Screen Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.5.4 Overall Opinion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

7 Conclusions and Perspectives 677.1 Lessons Learnt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

7.1.1 User Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677.1.2 User Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677.1.3 Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

7.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687.3 Future of the Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Contents v

A Illustration Cases 69A.1 Multi-Attribute Value Theory . . . . . . . . . . . . . . . . . . . . . . . . . 69

A.1.1 Decision Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69A.1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70A.1.3 Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70A.1.4 Performance Assessments . . . . . . . . . . . . . . . . . . . . . . . 71A.1.5 Value Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72A.1.6 Trade-offs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81A.1.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

A.2 ELECTRE III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87A.2.1 Decision Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87A.2.2 Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88A.2.3 Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89A.2.4 Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90A.2.5 Performance Assessments . . . . . . . . . . . . . . . . . . . . . . . 90A.2.6 Trade-offs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91A.2.7 Calculation of Concordance . . . . . . . . . . . . . . . . . . . . . . 92A.2.8 Calculation of Discordance . . . . . . . . . . . . . . . . . . . . . . 97A.2.9 Calculation of Credibility Score . . . . . . . . . . . . . . . . . . . . 100A.2.10 Ascending Distillation . . . . . . . . . . . . . . . . . . . . . . . . . 101A.2.11 Descending Distillation . . . . . . . . . . . . . . . . . . . . . . . . . 103A.2.12 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

A.3 Analytic Hierarchy Process . . . . . . . . . . . . . . . . . . . . . . . . . . 106A.3.1 Decision Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106A.3.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107A.3.3 Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108A.3.4 Objectives Comparison . . . . . . . . . . . . . . . . . . . . . . . . . 108A.3.5 Alternatives Comparison . . . . . . . . . . . . . . . . . . . . . . . . 115A.3.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

B Bisection Gap Calculation Algorithm Tests 121

Bibliography 133

Chapter 1

Introduction

“Freedom, after all, is simply being able to livewith the consequences of your decisions”

James X. Mullen

1.1 A Foreword about Decisions

As it was pointed out in [Pavesi 2000], people not only adapt themselves to theirsurrounding environment but they also intend to modify the universe. This is donethrough decisions, which may be either aimed at changing the current status ofthings —whenever a gap is perceived with regards to a desired potential situation—or might also intend to preserve a beneficial statu quo —protecting it from un-wanted modifications or perturbations—. On a similar direction, [Shackle 1966] hadexpressed that all human actions are linked to continuously transforming circumstances bydiscovering new meanings and new possibilities with regards to their physical environment.

Furthermore, [Pavesi 2000] emphasises the central role of decision makers inthe overall process. He notes that, even though they choose among alternatives, theydo so in terms of themselves, according to their vision of the World. Ultimately, theconsequences of their actions will be born by them —and normally by other peopletoo— but it is not only decision consequences that affect decision makers.On the contrary, the very same decision-making process has its impact too, as itmay affect aspiration levels, scales of preferences, the vision of the universe and person-ality. Pavesi even expresses: “In every choice, a decision maker dies and is born over again”.

Letting drama aside, beyond any shadow of a doubt, decisions do shape people’slives. The present is strongly determined by the decisions that were made in the pastbut, most importantly, current and future decision-making contexts are either empoweredor restricted by them. One of the key representatives of Argentine literature, Jorge LuisBorges, once wrote what might be translated as: “No decision is final. They all branchinto others” [Borges 1944, “The Lottery in Babylon”]. A millinery Chinese strategist andphilosopher, Sun Tzu, also illustrated this idea in her world-famous work about militarystrategy: “Opportunities multiply as they are seized ” [Tzu 1988].

To sum it up, decisions are inherent to human existence and development, theyare of utmost importance when it comes to people’s quality of life and, what is more,they feature a trace of interdependencies.

2 Chapter 1. Introduction

1.2 Main Complexities and Challenges of Decisions

1.2.1 Irreversibility and Opportunity Cost

Another aspect that further characterises the act of deciding is the fact that decisionmakers are expected to choose one and only one alternative because they start from aset of mutually exclusive items. As a result, deciding not only means picking onealternative but it also involves rejecting the others. This is linked to the concept of“Opportunity Cost” from Economic Sciences which, simply put, represents the value ofthe best alternative foregone.

In fact, as [Pavesi 2000] mentions, riskless decisions do not exist as the decisionmaker is “losing” the different potential worlds that the rejected alternatives might haveled to. The psychological burden of what is being lost is naturally a coercive aspectthat is present in any decision-making process that is worth of attention. Naturally,literature is no stranger to this: “Every time a man faces several alternatives, he choosesone and eliminates the others” [Borges 1944, “The Garden of Forking Paths”].

As far as important decisions are concerned, the degrees of freedom that arelost at the moment of making a decision are seldom likely to be recovered. Thisirreversibility is due to the fact that, once resources start being applied to move forwardinto the chosen direction, going backwards —if feasible— is extremely costly.

On top of that, [von Winterfeldt 1986] identifies five more difficulties that featurecomplex decisions: uncertainty, multiple conflicting objectives, multiple stakeholders, far-reaching consequences and environment dynamism, which will be briefly discussed as fol-lows.

1.2.2 Uncertainty

Uncertainty is linked to an unknown future but it should not be mistaken forpartial information . If we knew all values a variable could adopt and their expectedrelative frequency, we would have complete information about a situation of uncertainty.In most real-world scenarios, uncertainty is stressed by partial information.

As [von Winterfeldt 1986] mentions, we deal with uncertainty both rationally andirrationally. Whereas the latter involve ignoring it or simple worrying about it, theformer is associated with quantitative analyses, collecting evidence to reduce it orhedging against unfavourable outcomes, trying to find a balance between potential ben-efits and uncertainty.

1.2.3 Multiple Conflicting Objectives

All meaningful decisions involve analysing performances under multiple criteria.Even though different criteria might sometimes go together in the same direction, it isnot hard to come across with cases where doing well on one criterion requires doingpoorly on another. In those cases, that are the general rule rather than the exception,

1.2. Main Complexities and Challenges of Decisions 3

a trade-off is necessary.

“Trade-offs are judgements linked to the decision maker’s assessment of the relativedesirability of the available options on each dimension and on his or her feelings aboutthe relative importance of these dimensions” [von Winterfeldt 1986].

1.2.4 Multiple Stakeholders

Some works as [Pavesi 2000] consider that the decision-maker is always a singleperson or, in other words, a unique and indivisible entity. Under this reasoning, groupdecisions are eventually the accumulation of individual decisions. This assumes theexistence of a social preference or welfare function that translates individual decisionsinto a decision abided by everyone.

Notwithstanding, whether multiple decision makers are involved or not, even themost personal decisions affect multiple stakeholders. In simple terms, a stakeholderis any person that might affect or get affected by the decision. It is importantto bear in mind that our decisions not only affect us but they also affect otherpeople and, as such, they also either empower or restrict someone else’s decisions contexts.

Different people might naturally identify different objectives or criteria for thedecision. They might have different trade-off expectations and they might also assessalternatives performances, uncertainties and risk differently.

In Economics, moral hazard occurs when “people or organizations do not suf-fer from the results of their bad decisions, so may increase the risks they take”[Cambridge University Press ]. This has been the underlying reason of several of the keyfinancial crises that the World has witnessed. Even without reaching the extreme pointof moral hazard, as it was pointed out before, the simplest personal decisions mayalso affect other people and that is the reason why they need to be considered in thedecision making process.

In particular, groups of stakeholders may be considered and prioritised accordingto different dimensions such as power, interest, need and support.

On the other hand, [Munda 2004] proposes the concept of social multi-criteria eval-uation (SMCE) as a possible useful framework for the application of social choice tothe difficult policy problems. The foundations of SMCE are set up by referring toconcepts from complex system theory and philosophy, such as reflexive complexity,post-normal science and incommensurability.

1.2.5 Far-reaching Consequences and Environment Dynamism

The longer the time horizon of the decision and the more dynamic the contextenvironment are, change becomes more and more likely.

4 Chapter 1. Introduction

For [von Winterfeldt 1986], anticipating how and when change will occur is key forpreparing for the unseen and being able to adapt effectively.

1.3 General Goals and Structure of the Document

The goals of this project are manifold. To start with, this Thesis has the objective offurther elaborating the ideas presented in the state of the art with regards to methodchoice in Multi-Criteria Decision Analysis (MCDA). Chapter 2 is devoted to thispart of the work.

Secondly, on the basis of the aforementioned theoretical contribution and, after iden-tifying and considering the advantages and disadvantages present in non-commercialand commercial decision applications (Chapter 3), a new multi-method MCDAsoftware application will be delivered, whose design and implementation are explainedin Chapter 4. The application will be web-based and multilingual. On the other hand,it will be made available online through a self-maintained rented server.

Furthermore, the project intends to democratise the access to decision aiding.While this undoubtedly delivers a social contribution, it might also lead the creationof a new market —the “retail” decision-aiding market—, whose potential profitabilityis worth being analysed.

In order to be able to achieve the aforementioned goals, a new company willbe created under the laws of the French Republic. The administrative and legaldimensions of this will also be discussed as part of this work (in Chapter 5). This mighthopefully pave the way for future IT4BI specialisation students in CentraleSupélec whomight be interested in orienting their theses in the same direction as this project is heading.

Then, Chapter 6 presents the feedback that was received from the public as theytried the new software solution. Subsequently, Chapter 7 presents the Conclusions andFuture Perspectives of the project.

Appendix A is an important one, since it presents three illustration cases developedin detail to depict the particularities and complexities of the methods that have beendeveloped so far.

Finally, Appendix B contains the evidence of the tests performed to assess the algo-rithm we defined for the computation of gaps in the Bisection Method as explained in4.1.2.2.

1.4. Elements of Innovation 5

1.4 Elements of Innovation

The novel approach for MCDA method choice that this project aims to deliver isspecifically targeted to personal and business ends-users who are not necessarilyexperts in the MCDA field.

The latter is quite a remarkable point not only from a theoretical perspective butalso from a pragmatical perspective, since it enables the development of a self-servicedecision-aiding software solution for the general public that features multipleMCDA methods and guides users to choose the most suitable one according to theparticularities of his or her decision-making context. As it will be illustrated in Chapter3, this has never been seen in other software solutions.

Notwithstanding, guidance is not just meant to be provided during the MCDAmethod choice. On the contrary, it will be provided all over the process, starting fromthe elicitation of objectives and alternatives —two crucial steps of the decision structur-ing phase that are common to all methods— and following with the other subsequent steps.

Furthermore, particular contributions will be provided when it comes to the method-dependant steps. As it will be explained in detail in Section 4.1, novel and interestingapproaches will be put in place, for example, as far as the following points are concerned:

• Different options for Value Functions customisation, including a graphical im-plementation of the Bisection Method which backed by a custom algorithm toautomatically calculate interval adjustments (for MAVT method),

• Graphical Swing Weights elicitation (also within MAVT method),

• Implementation of Roy’s revised version of Simo’s white cards procedure (as far asELECTRE III method is concerned),

• Robustness Assessment through Monte Carlo Methods (for MAVT and AHPmethods).

Chapter 2

Multi-Criteria Decision AnalysisMethods

2.1 Decision Aiding

As defined by [Roy 2005], Decision Aiding is the activity involving the use ofexplicit models —though they do not necessarily have to be completely formalised—,to help obtain answers to the questions posed by a stakeholder in a decision process.

These answers work towards clarifying the decision and usually involve recom-mending, or simply favouring, a behaviour that will increase the consistency with thestakeholder’s objectives and value system.

[Roy 2005] also mentions what might be reasonably expected from Decision Aiding:

• Analysing the decision making context.

• Organizing and/or structuring how the decision making process unfolds in orderto increase coherence.

• Getting the actors to cooperate through an environment of debate and mutualunderstanding.

• Elaborating recommendations using results taken from models and computa-tional procedures.

• Participating in the final decision legitimisation.

2.2 Multi-Criteria Decision Aiding (MCDA)

MCDA is a general framework for supporting complex decision-making situationswith multiple and often conflicting objectives that stakeholders groups and/ordecision-makers value differently.

Following the reasoning from [Roy 2005], Multi-Criteria approaches:

• Delimit a broad spectrum of points of view.

• Construct a family of criteria which preserves, for each of them, without anyfictitious conversion, the original concrete meaning of the corresponding evalu-ations.

8 Chapter 2. Multi-Criteria Decision Analysis Methods

• Facilitate debate on the respective role that each criterion might be called uponto play during the decision aiding process.

.

2.3 The Need for Choosing Among Multiple MCDA Meth-ods

Naturally, all MCDA methods have advantages and disadvantages whoseimportance depends on the particularities of each decision-making scenario.

In order to be able to cope with the diversity and complexity of real-worlddecisions, a heterogeneous set of methods is required. It might be normal for people tofeel biased towards a homogeneous set of methods but this should be avoided to thegreatest extent that natural subjectivity allows.

As “one-size-fits-all” solutions do not exist, no method is adequate to tackle allpossible decision-making situations, in spite of the fact that some software solutions seemto assume this. That is the reason why a novel and scalable approach will be proposedto assess each scenario on a case-by-case basis and choose the most suitable methodaccordingly.

2.4 A High-Level Overview of MCDA Methods

There is a broad range of MCDA Methods Classifications. We will briefly discusstwo of them in order to provide the reader with a high-level overview of what mightbe found in the universe of MCDA Methods.

One of the classification is presented by [Figueira 2005a, Page 906] and will be coveredin subsection 2.4.1. The second one is proposed by [Dias 2004] and will be covered in2.4.2.

2.4.1 School-Based Classification

In [Figueira 2005a, Page 906], two major schools of thought are differenti-ated: the Multi-Attribute Utility (or Value) Theory and the Outranking Approach. In[Santiteerakul 2012], the first category receives a more generic name —Single synthesizingcriterion methods— and a third category is introduced: Programming methods.

Following the second classification, Single synthesising criterion methods havethe following characteristics:

• As it is indicated by the name of the category, a unique criterion is obtained at theend of the process, which determines a complete pre-order. This is done throughthe application of mathematical formulas.

2.4. A High-Level Overview of MCDA Methods 9

• Transitivity of preferences and indifferences is an important axiom.

• It is assumed that each alternative shows a well-defined degree of satisfactionfor each criterion (namely, no thresholds are accepted).

• Uncertainty is dealt with through probability distribution, fuzzy numbers orrough sets theory.

• Some methods within this group include:

– Multi-Attribute Value Theory (MAVT ),

– Multi-Attribute Utility Theory (MAUT ),

– Simple Multi-Attribute Rating Technique (SMART ),

– Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS ),

– Measuring Attractiveness by a Categorical Based Evaluation Technique (MAC-BETH ), and

– Analytic Hierarchy Process (AHP).

Secondly, Outranking methods present these particularities:

• They are based in weaker assumptions. It does not require transitivity, forinstance.

• No compensation of contradictory performances in different criteria is allowed.

• Uncertainty is managed through the use of preference and indifference thresh-olds. Some models also include veto thresholds.

• Some methods within this group include:

– Elimination and Choice Expressing Reality —“Elimination et Choix Traduisantla Realité”— (ELECTRE ),

– Preference Ranking Organization Method for Enrichment Evaluations(PROMETHEE ),

– Novel Approach to Imprecise Assessment and Decision Environments (NA-IADE ), and

– “Regime” Method.

Finally, the category of Programming methods belongs in the general category ofContinuous methods —as opposed to Single synthesizing criterion methods and Outrank-ing methods, which belong in the general category of Discrete Methods—. In this case, thechoice is not to be done among a set of finite mutually exclusive alternatives. On the con-trary, alternatives are generated during the solution process on the basis of a mathematicalmodel formulation. Some examples of this third category are:

• Multi-Objective Programming (MOP), and

• Goal Programming (GP).


2.4.2 Approach-Based Classification

[Dias 2004] presents a classifications that divides methods according to the approachthey follow: normative, descriptive, prescriptive or constructive.

Normative approaches establish norms a priori and derive rationality out ofthem. Those norms are postulated as a requirement for rational behaviour.

Descriptive approaches derive rationality models from observing how decisionmakers behave. They tend to link the way decisions are made with the quality ofthe outcomes.

Prescriptive approaches discover rationality models for a given decision makerfrom his or her answers to preference-related questions.

Finally, Constructive approaches build rationality models for a given decisionmaker from his or her answers to preference-related questions.

2.5 Methods Implemented during the Initial Phase

As it had been planned in the Master’s Thesis Proposal document, a subset ofmethods was chosen as a starting point. In particular, three methods were chosen to beimplemented at this stage and their choice was not only due to their mass worldwideutilisation but also because of their diversity.

Those methods are Analytic Hierarchy Process (AHP) (see subsection 2.5.1),ELECTRE III (see subsection 2.5.2) and Multi-Attribute Value Theory (MAVT) (seesubsection 2.5.3). As it was discussed in Section 2.4, MAVT and AHP belong in thefamily of Single synthesizing criteria methods which, among other aspects, compensatecontradictory performances and produce a numeric score for each alternative. On thecontrary, ELECTRE III is an Outranking method, which does not allow compensationand outputs an ordinal ranking.

Even though MAVT and AHP belong in the same category, they have four importantdifferences:

• Firstly, MAVT gathers individual performance assessments for the alternativesas either numeric values (e.g. “the Price of Car 1 is 9,000 EUR” and “the Priceof Car 2 is 20,000 EUR.”) or categorical values (e.g. “the Quality of Car 1 is«Normal»” and “the Quality of Car 2 is «Good»”).

On the contrary, AHP gathers pairwise qualitative comparisons such as “Car 1is strongly better than Car 2 as far as Price is concerned” or “Car 2 is moderatelybetter than Car 1 as far as Quality is concerned” (the model does not even explicitlyrequire the exact values for Price or Quality).

2.5. Methods Implemented during the Initial Phase 11

• Secondly, AHP also obtains criteria weights through pairwise comparisons (e.g.“Quality is moderately more important than Price”), regardless of the ranges of valuesactually seen in alternatives.

At variance with that, MAVT elicits the so-called swinging weights, which takeinto account the difference between the best and the worst alternatives as far as eachcriterion is concerned. Therefore, instead of measuring the intrinsic importance ofeach criterion in abstractum, ranges of values are taken into account.

• Thirdly, the process to obtain results in the MAVT may be intuitively explainedto a public with no technical background in MCDA. However, AHP is not as trans-parent as MAVT is, since it involves steps like Eigenvalues calculation that non-technical audiences probably will not master and will therefore consider the methodas a black-box.

• Fourthly, MAVT is independent to third alternatives. Consequently, addingalternatives will not cause a rank reversal as it might happen with AHP.

As it was shown, these methods exhibit differentiating characteristics that makethem advisable under different decision-making contexts. It should be noted that, eventhough this selection sets a good starting point, more methods are planned tobe implemented in the future because, as it will be explained in Chapter 7, thisinitiative is going to continue after the delivery of this Master’s Thesis.

The next subsections present a very general overview of AHP, ELECTRE III andMAVT in particular and provide references to the relevant specialised works.

2.5.1 Analytic Hierarchy Process (AHP)

As defined in [Saaty 2005] the Analytic Hierarchy Process (AHP) is a theory ofrelative measurement of intangible criteria.

In traditional measurement, elements are measured one by one, not by comparingthem with each other. With the approach to relative measurement, a scale of prioritiesis derived from pairwise comparison measurements only after the elements to bemeasured are known.

In the AHP paired comparisons are made with judgements using numericalvalues taken from the AHP absolute fundamental scale of 1-9. A scale of rela-tive values is derived from all pairwise comparisons and it also belongs to an absolute scale.

AHP is useful for making multi-criteria decisions involving benefits, opportunities,costs and risks.

A full illustrated example of AHP may be found in A.3.


2.5.2 ELECTRE III

As [Figueira 2005a] summarises, ELECTRE methods are relevant when there are be-tween 3 and 12 or 13 criteria. That being said, ideally there should be at least 5criteria. In addition, at least one of the following situations must be true:

• Actions are evaluated (for at least one criterion) on an ordinal scale or on a weaklyinterval scale.

• A strong heterogeneity related with the nature of evaluations exists among cri-teria.

• Compensation of the loss on a given criterion by a gain on another one may notbe acceptable.

• For at least one criterion, small differences of evaluations are not significantin terms of preferences, while the accumulation of several small differencesmay become significant (this is modelled through the introduction of thresholds).

The foundations of ELECTRE methods were set by [Roy 1990]. As defined in[Figueira 2005b], there are different variants of ELECTRE for different purposes:

• Choice Problematic: ELECTRE I, ELECTRE Iv and ELECTRE IS.

• Ranking Problematic: ELECTRE II, ELECTRE III and ELECTRE IV.

• Sorting Problematic: ELECTRE TRI.

Let us now go a little bit deeper into those methods linked to the ranking problem-atic:

• ELECTRE II: This method was the first ELECTRE method designed to deal withranking problems. It was also the first method to use a technique based on theconstruction of an embedded outranking relations sequence.

• ELECTRE III: ELECTRE III was designed to improve ELECTRE II and thus dealwith inaccurate, imprecise, uncertain or indetermination of data.

• ELECTRE IV: ELECTRE IV is also a procedure based on the construction of a setof embedded outranking relations and its exploiting procedure is the sameas in ELECTRE III, but this method is designed to manage the impossibility ofacquiring preference information about relative importance of criteria.

A full illustrated example of Electre III may be found in A.2.

2.5.3 Multi-Attribute Value Theory (MAVT)

As described in [van Herwijnen 2010], MAVT can deal with problems with a finiteand discrete set of alternatives that have to be evaluated on the basis of conflictingobjectives.

2.6. Other Works on MCDA Method Choice 13

For all objectives, one or more different attributes are used to measure performance.As these performances are normally expressed in different measurement scales, ValueFunctions are used to translate them into standardised scores. Then, those scores can beaveraged to get a unique score for each alternative and the best alternative is the onehaving the maximum score.

Therefore, MAVT involves four steps1:

1. Selection and definition of criteria: identify the effects or indicators relevant for thedecision.

2. Definition of alternatives (the mutually exclusive options that are to be comparedto each other).

3. Assessment of scores for each alternative in terms of each criterion.

4. Ranking of the alternative.

The first three steps are the same as in most MCA methods but the fourth step is spe-cific for MAVT. The theoretical foundations of MAVT were described by [Fishburn 1967]and [Keeney 1993].

A full illustrated example of MAVT may be found in A.1.

2.6 Other Works on MCDA Method Choice

The work of [Guitouni 1998] constitutes one of the first steps for proposing a method-ological approach to select an appropriate MCDA method to a specific decision-makingsituation.

Quite a few years after, [Roy 2013] outlined a hierarchical approach to MCDA MethodChoice. Firstly, a so-called crucial question is posed, asking what type of results areexpected by the decision maker. This question, that we will refer to as Q1, has five possibleanswers:

• Type 1: A numerical value (utility, score) is assigned to each potential action. Thisrestricts methods to MAVT, MAUT, UTA, MACBETH, AHP, SMART, TOPSIS,Choquet Integral, UTAGMS, Fuzzy AHP and Fuzzy TOPSIS.

• Type 2: The set of actions is ranked (without associating a numerical value toeach of them) as a complete or partial weak order. This leads the direction towardsELECTRE III and IV, PROMETHEE I and II, Robust Ordinal Regression methods,the Dominance-based Rough Set approach or the Machine Learning approach.

1Following “Value-Focused Thinking”, criteria are defined before alternatives, whereas following“Alternative-Focused Thinking


• Type 3: A subset of actions, as small as possible, is selected in view of a final choiceof one or, at first, few actions. Some methods for this case are ELECTRE I and IS,PROMETHEE V and Rubis, among others.

• Type 4: Each action is assigned to one or several categories, given that the set ofcategories has been defined a priori. Relevant methods include Dominance-basedRough Set Approach, UTADIS, PREFDIS, UTADISGMS, ELECTRE TRI-B, TRI-C, and TRI-NC, the filtering method, PROAFTN, TRINOMFC, PAIRCLASS andTHESEUS, among others.

• Type 5: A subset of potential actions enjoying some remarkable properties is pro-vided to serve as a base in the following stage of the decision aiding process. Thisincludes procedures based on the exploration of the alternatives set and proceduresbased on the progressive contraction of the alternatives set.

Following with the approach defined in [Roy 2013], only after the first question hasbeen answered, may other five key questions be asked. They are:

• Q2) Is the method able to use original performance scales or able to transform themin a meaningful and non-arbitrary way?

• Q3) How difficult is it to get preference information that the method requires?

• Q4) Should the part of imprecision, uncertainty or indetermination in the definitionof performances be taken into account?

• Q5) Is compensation of bad performances on one criteria by good ones on othercriteria acceptable?

• Q6) Is it necessary to take into account some forms of interaction among criteria?

Finally, the authors point out three so-called secondary questions:

• Q7) Is the method able to satisfy properly the needs of comprehension from the partof stakeholders involved in the decision process?

• Q8) Is an axiomatic characterization of the method available, and if so, is it accept-able in the considered decision context?

• Q9) Can the weak points of the method affect the final choice?

Figure 2.1 is not included in [Roy 2013] but it has been elaborated to visually ex-plain the sequential nature of the approach and the interdependence among the differentquestions.

2.6. Other Works on MCDA Method Choice 15

Figure 2.1: Flowchart of own elaboration to illustrate the first question from [Roy 2013]


2.7 Introducing the “Parliamentary” Approach

On the basis of a subset of questions defined by [Roy 2013] and summarised inSection 2.6, we would like to propose a different approach that presents three maindifferences: it is non-hierarchical, it intends to reduce ambiguity and complexityand, at the same time, it primarily prioritises non-technical aspects that can be an-swered by the general public, while leaving room for further scaling through additionalquestions in the future. Each of these aspects will be discussed in the following subsections.

2.7.1 Non-hierarchical Nature

To start with, instead of turning to a hierarchical or sequential strategy, our approachevaluates answers in a parallel way and we would like to express the reasons thatmotivate this design decision.

The results that can be achieved on a particular situation naturally depend onrequirements but they are also determined by the available means. Let us illustratethe aforementioned idea with a metaphor: imagine that a family have to choose aninvestment type in order to capitalise their savings. If we start by asking them whattheir expected result is, they might reply, for example, that they would like to obtaina return on the investment of 20 %. At first sight, that might suggest that stocks andfutures could be feasible solutions to be advised. However, if you subsequently askthem what time horizon they envision for their investment, they might reply with ashort time period, which would be colliding with the advice of investing in stock or futures.

That means that either the family has to either change their minds regarding require-ments (for instance, the time horizon) or adapt the expected result (i.e. the return oninvestment rate they want to obtain) to the other circumstances. In more abstract terms,requirements might be colliding with each other and such a conflict needs tobe confronted and put in evidence as a previous step to be able to find a compromise.

Let us turn to a second metaphor. One of the foundational concepts of traditionalProject Management discipline is the Time-Cost-Scope triangle. This is also known asthe triple constraint2 and tries to emphasise that any change in one of the factors needs tobe followed by a change in at least one of the remaining factors. For instance, no ProjectManager can add deliverables to the scope of a project without negotiating an increase ofthe budget and/or on the schedule. Similarly, if a delivery date is moved backwards, costswill probably need an increase so as to bring additional staff (also known as “crashing”).If stakeholders do not understand this, it is the duty of the Project Manager to manageexpectations according to feasibility.

Expectations management also applies to decision-making onsets: decision mak-ers who are not experts in MCDA do not know the particularities of each

2Some authors even refer to an hepta-restriction that also comprises Quality, Risks, Materials andCustomer Satisfaction but let us keep the three-variable version for our example.

2.8. How Selection Works in the “Parliamentary” Approach 17

method so we need to provide them with the relevant guidance so that they can ac-knowledge and understand what is possible and what is not possible in each case.

The purpose of these two metaphors is to bring awareness to the fact that the type ofexpected results is not necessarily a variable that is independent from the othercharacteristics of the decision-making scenario.

What happens if the decision maker explains that he or she expects numerical scores tobe assigned to each alternative but then indicates that it is hard for him or her to explicitpreference information about the relative importance of criteria? Should the analyst stillconsider Electre III, for its ability to cope with a lack of such preference information orshould he or she discard it because it only provides results in a complete or partial weakorder? Should the analyst check if the result type is negotiable? Maybe it turns out tobe non-negotiable and the adjustment is eventually done on the preference elicitation sideinstead but at least it is worth checking.

2.7.2 Reducing Ambiguity and Complexity

Whenever possible, the design of questions will try to reduce the number of possibleanswers to two or three in order to facilitate the assessment process to end-users. Forexample, as far as the answers of “Q1 ” are concerned, we consider that “Type 4 ” may wellencompass “Type 3 ” and “Type 5 ”, as either the elimination of alternatives or the pre-selection of a subset of potential actions enjoying remarkable properties may eventuallybe conceived as a Categorisation problems.

2.7.3 Prioritising Questions

In the same vein as we prioritised a subset of MCDA methods to tackle during thefirst phase of this project, we will also prioritise questions.

The main goal is to focus on relevant and non-technical aspects that may beanswered by the general public, at the same time we ensure that questions are discrimi-natory enough with regards to the MCDA methods that will be implemented.

The non-hierarchical nature of the approach is what provides the ability to easilyscale, as new questions can be added in a cleaner and simpler way when no interdepen-dence needs to be taken care of.

2.8 How Selection Works in the “Parliamentary” Approach

As we said before, the method consists of a number of questions regarding thedecision-making scenario. The answers to those questions end up being translated as“Approvals”, “Vetos” or “Abstentions” as far as each method is concerned, in the samevein as it happens in legislative or international bodies.


The first version3 of the set of questions and their answers is detailed as follows.Please notice that, as it was noted before, the order is not relevant at the moment ofassessing answers.

1. What would you like to obtain at the end of the process?

(a) A numerical score for each of the options I am choosing among.

(b) A ranking of the options will be enough (e.g. option "C" is the best, option"A" is the second best, etc.).

(c) I only want to assign each option to one or more categories (e.g. "Recommend-able options", "Options to be rejected", etc.).

This question is linked to “Q1” from the original questionnaire, which refers to theoutput of the decision-making process.

Answer “1a” is linked to “Type 1” results. Therefore, among implemented methods,this answer will favour AHP and MAVT but it will penalise ELECTRE III.

Answer “1b” is associated with “Type 2” results, whereas Answer “1c” would bemapped to “Type 3”, “Type 4” and “Type 5” (as explained in subsection 2.7.2). Both“1b” and “1c” are compatible with the three implemented methods.

2. If an option is very good in one aspect but very bad in another aspect,would you like to make compensations?

(a) Not at all.

(b) Maybe.

(c) Yes, they should be compensated.

This question is linked to “Q5” from the original questionnaire and is meant to chooseamong compensatory and non-compensatory methods.

Answer “2a” rewards the non-compensatory method, that is ELECTRE III, whileit penalises AHP and MAVT because they are compensatory. In turn, Answer “2c”has the opposite effect.

Answer “2b” originally meant to provide a neutral choice, for those users that con-sidered that compensation was a desirable but not indispensable feature. However,after long debate, it was eventually decided that it will be removed in the secondversion of the questionnaire so as to minimise ambiguity.

3. In which way would you like to define your preferences?

(a) I would like to do comparisons such as "X is strongly better in Quality thanY", "they have the same Comfort", etc.

(b) I would like to define all my preferences directly with numbers.3Following meetings we held with international experts in the subject of MCDA and, considering the

collected feedback, a second version of the questionnaire is due to be designed and implemented inElectioVis by September 2015.

2.8. How Selection Works in the “Parliamentary” Approach 19

(c) I would like to use both categories ("Good", "Bad", etc.) and numbers.

This question is linked to “Q3” from the original questionnaire and is related to theway preferences are gathered.

Answer “3a” is consistent with the qualitative assessments found in AHP, so thismethod gets rewarded. The other two methods get penalised as they are not basedin pairwise comparisons.

Answer “3b” rewards AHP because pairwise comparisons can also be expressed withnumbers ranging from 1 to 9. It also rewards MAVT, which is compatible with nu-merical and categorical performance assessments. However, it penalises ELECTREIII, because the latter is advised for cases where there is at least one qualitativecriterion.

Answer “3c” rewardsMAVT and ELECTRE III because both of them allow numericand categorical performance assessments but it penalises AHP.

4. How stable is the list of options from where you are choosing?

(a) I am aware that rankings might eventually change and I am fine with that.

(b) The ranking of the unchanged alternatives should necessarily keep consistency.

This question is somehow linked to “Q9” from the original questionnaire as the riskof rank reversal might well be one of the weaknesses of a method. Instead of deemingthis question as a secondary one, we consider that it is important to be included inthe model.

Answer “4b” rewards MAVT but penalises AHP and ELECTRE III because theyare subject to the risk of rank reversal.

Answer “4a” rewards MAVT too but it abstains as far as the other two methods areconcerned, because the user is indicating that he or she is not concerned about rankreversals.

Table 2.1 summarises assessments for each answer and method.


Table 2.1: Assessments by Answer and MethodAHP ELECTRE MAVT

1) What would you like to obtain at theend of the process?1a) A numerical score for each of the options Iam choosing among.

Approval Veto Approval

1b) A ranking of the options will be enough (e.g.option "C" is the best, option "A" is the secondbest, etc.).

Approval Approval Approval

1c) I only want to assign each option to oneor more categories (e.g. "Recommendable op-tions", "Options to be rejected", etc.).

Approval Approval Approval

2) If an option is very good in one aspectbut very bad in another aspect, would youlike to make compensations?2a) Not at all. Veto Approval Veto2b) Maybe. Approval Abstention Approval2c) Yes, they should be compensated. Approval Veto Approval3) In which way would you like to defineyour preferences?3a) I would like to do comparisons such as "X isstrongly better in Quality than Y", "they havethe same Comfort", etc.

Approval Veto Veto

3b) I would like to define all my preferences di-rectly with numbers.

Approval Veto Approval

3c) I would like to use both categories ("Good","Bad", etc.) and numbers.

Veto Approval Approval

4) How stable is the list of options fromwhere you are choosing?4a) I am aware that rankings might eventuallychange and I am fine with that.

Abstention Abstention Approval

4b) The ranking of the unchanged alternativesshould necessarily keep consistency.

Veto Veto Approval

Chapter 3

A Survey on Current MCDASoftware Solutions

“[...] it is tempting, if the only tool you have is ahammer, to treat everything as if it were a nail”

Abraham Maslow

As a result of all of the aforementioned complexities, it is indisputable that the aidof a Decision Support System is not only advisable but also mandatory. If we lookat products currently available in the market, we will see that there are economic,cognitive and language barriers that forestall the access of the general public tothem.

Whereas other open-source solutions do not pose problems from the economicperspective, they normally do not excel in terms of interactivity and user friendli-ness. Such products are normally developed focusing on academic environments and aretherefore targeted towards technical audiences.

On the other hand, several commercial solutions tend to be good in the latteraspects but they are still out of reach of the general public because of economicreasons. They are normally accessible only to companies who can afford licensing costs.

On top of that, available solutions normally deal with one or two Multi-CriteriaDecision Aiding methods, in spite of the fact that we can find more than fortymethods in the relevant bibliography. And, whenever more than just one methodis offered, there is no guidance for users to choose the most suitable one accordingto his or her decision-making context. This might well be a reminder of Maslow’s law ofthe hammer [Maslow 1966].

Let us now discuss the particularities of available tools —both commercial andnon-commercial—.

3.1 Non-Commercial Solutions

As it will be discussed in this subsection, the following limitations could be pointedout as far as the available non-commercial solutions are concerned:

• Lack of appropriate on-screen guidance.

22 Chapter 3. A Survey on Current MCDA Software Solutions

• Whenever more than one MCDA method is offered, the user has to make achoice by himself or herself as no help is provided.

• Some input screens are extremely technical or mathematically-oriented.

• Some of the interfaces are text-only or based on obsolete Graphic User Inter-face (GUI) technologies, making them unfriendly.

• Most of the solutions present scarce or non-existent interactivity possibilities.

• In several cases, the date of the last released version poses doubts about the activemaintenance of the solutions.

• Apart from very few exceptions, most solutions are monolingual.

In some cases, the application is explicitly targeted to teachers and researchers but,for the aforementioned reasons, one might also reasonably assume this orientation for theremaining cases.

Table 3.1 provides a summary of the key features of non-commercial Multi-Criteria Decision Aiding solutions mentioned in[EURO Working Group on Multicriteria Decision Aiding ]. Applications with non-existent or invalid URL links have been excluded from analysis.

Table 3.1: Non-commercial Software SolutionsName Target MCDA Methods Lang. Latest release

diviz Academics

ACUTA, Electre,Promethee, UTA,UTASTAR, MAVT,Weighted Sum, etc.

EN, FR May 2015 (1.14)

GMAA Academics MAUT EN 2003 (1.01)iMOLPe Academics Linear Programming EN 2012 (2.1)jMAF Academics DRSA, VC-DRSA EN May 2008 (1.0)jRank Academics DRSA, VC-DRSA EN October 2013JSMAA Academics SMAA EN January 2015 (1.0.3)MCDA-ULaval

Academics Electre EN, FR April 2015 (0.6.1)

WWW-NIMBUS

Academics Optimisation EN July, 2013 (4.1.2)

The following paragraphs provide further details about each case.

3.1. Non-Commercial Solutions 23

3.1.1 diviz

• Developed by: Institut Telecom — Telecom Bretagne (France).

• Website: http://www.decision-deck.org/diviz/index.html.

• Platform: Java application (requires installation).

• Licence type: CeCILL (French Free Software Licence developed by the Commis-sariat à l’Énergie Atomique (CEA), the Centre National de la Recherche Scientifique(CNRS) and the Institut National de Recherche en Informatique et Automatique).

• Additional comments:

– This is the most actively maintained project within the cases analysed.

– Several MCDA methods are offered but no guidance is provided to choose themost suitable one. It is assumed that the end-user is an expert who alreadyknows which method is to be applied.

– A project is built as a workflow, by dragging components to a canvas and in-terconnecting them. The website of the project contains some sample projectsbut the application contains no on-screen help to facilitate the choice and ar-rangement of components. Technical names are frequently shown and the userneeds to be able to deal with them.

– Interactivity is scarce. In order to experiment with different values, for instance,the user needs to modify the input files (in CSV or XML format) and re-executethe process.

– Despite requiring installation, the software cannot work in an autonomous waysince webservice invocations are required.

Figure 3.1: Example of a screenshot from diviz

http://www.decision-deck.org/diviz/index.html


3.1.2 GMAA

• Developed by: Technical University of Madrid (Spain).

• Website: http://www.dia.fi.upm.es/~ajimenez/GMAA.

• Platform: Windows Application.

• Licence type: Proprietary (no source code is available).


– Alternative assessments may be defined as intervals, which is an advantage inscenarios where imprecision is present.

– Utility functions may be defined in a wide variety of ways, which is a verygood point too. That being said, this might eventually confuse users withouta technical background, since accessible on-screen guidance is not provided.

– Weight elicitation based on trade-offs is available but it does not provide graph-ical representations and the user ends up doing abstract comparisons betweennumbers.

– Sensitivity Analysis also offers a wide range of features.

– The application is not being actively maintained. The latest version is from2003.

Figure 3.2: Example of a screenshot from GMAA

3.1.3 iMOLPe

• Developed by: University of Coimbra (Portugal).

• Website: http://www.uc.pt/en/org/inescc/products.

• Platform: Windows Application (developed with Delphi).


http://www.dia.fi.upm.es/~ajimenez/GMAA

http://www.uc.pt/en/org/inescc/products



– The documentation shows several charting features.

– The sequence of data input is explained in the help file but is not enforced withon-screen indications.

– Data is not validated on input but afterwards, which causes unexpected errorswhich are difficult to troubleshoot.

– The installer offers four languages (English, Spanish, Portuguese and French)but the application only works in English.

Figure 3.3: Example of a screenshot from iMOLPe

3.1.4 jMAF

• Developed by: Laboratory of Intelligent Decision Support Systems — Poznan Uni-versity of Technology (Poland).

• Website: http://www.cs.put.poznan.pl/jblaszczynski/Site/jRS.html.

• Platform: Java stand-alone application.

• Licence type: Proprietary (no source code is available). Terms and conditions pro-hibit any utilisation that is not linked to research, education or non-profit privateuse.


http://www.cs.put.poznan.pl/jblaszczynski/Site/jRS.html


– The interface is based on Eclipse, which will probably be familiar to developersbut might not be as accessible for users without a technical background.

– The application offers two methods but the end-user must choose one of themwithout any guidance.

– The latest version is more than 7 years old, which poses doubts about themaintenance continuity of the project.

Figure 3.4: Example of a screenshot from jMAF

3.1.5 jRank

• Developed by: Institute of Computing Science — University of Poznan (Poland).

• Website: http://www.cs.put.poznan.pl/mszelag/Software/jRank/jRank.html.

• Platform: Java console application (no Graphic User Interface).



– The application is meant to be run in a console environment which impedesany graphical representations and restricts all outputs to texts.

– It is assumed that input files (with “ISF” extension) are prepared externally.

– Even though it is explained in the "readme" file, the application requirescommand-line skills (creating and changing directories, etc.). This is attain-able for an audience with a background in Computer Science but it makes theapplication distant from the general public.

http://www.cs.put.poznan.pl/mszelag/Software/jRank/jRank.html


Figure 3.5: Example of a screenshot from jRank

3.1.6 JSMAA

• Developed by: Tommi Tervonen et al (Netherlands).

• Website: http://www.smaa.fi/jsmaa.

• Platform: Java stand-alone application.

• Licence type: GNU General Public License Version 3.


– The application implements Stochastic Multicriteria Acceptability Analysis,including:

∗ SMAA-2 for Multi-Attribute Value/Utility Theory based decision analysis(ranking / choosing problem statement).

∗ SMAA-TRI for outranking based sorting problems.∗ SMAA-O for ordinal criteria measurements (part of SMAA-2-model).

– No guidance is provided to users so as to choose the most suitable model.

– Performance assessments may be input in a wide variety of ways, which is avery good point. For instance, they may be entered as:

∗ Exact Values.∗ Intervals.∗ Gaussian distributed measurements.∗ LogNormal distributed measurements.

http://www.smaa.fi/jsmaa


∗ LogitNormal distributed measurements.∗ Beta distributed measurements.∗ Discrete distributed measurements.

– The interface is very easy to follow. That being said, context menus mightfacilitate navigation even more but they are missing.

– Alternatives and criteria cannot be renamed, so the user need to work with“Alternative 1”, “Alternative 2”, ..., “Criteria 1”, “Criteria 2”, ..., etc.

– Result may be visualised in two forms: Rank acceptability indices (bar chart)or Central Weight Vectors (line chart). Both of them may be exported asGNUPlot scripts.

Figure 3.6: Example of a screenshot from JSMAA

3.1.7 MCDA-ULaval

• Developed by: Monade laboratory — University of Laval (Canada).

• Website: http://cersvr1.fsa.ulaval.ca/mcda.

• Platform: Java stand-alone application (executable JAR).



– The application allows different types of sensitivity analysis, which representsquite an advantage.

– It also has an option for modelling Interaction between criteria, which is rarelyseen on other pieces of software.

http://cersvr1.fsa.ulaval.ca/mcda


– It is possible to visualise intermediate results of calculations, thus giving sometransparency to final rankings.

– Even though it allows multiple methods among Electre family, other approachesare excluded.

– No guidance is provided to choose the most suitable Electre method.

– A manual is available but no on-screen instructions are provided to help userswhile completing parameters.

– The application is targeted to experts. Many parts of the interface have atechnical level that is far from the general public. For instance, some parametersare denoted with Greek letters instead of natural language.

Figure 3.7: Example of a screenshot from MCDA-ULaval

3.1.8 WWW-NIMBUS

• Developed by: Department of Mathematical Information Technology — Universityof Jyväskylä (Finland).

• Website: https://wwwnimbus.it.jyu.fi.

• Platform: Web application (developped in Python).


• Reference: [Miettinen 2006].


– The user is asked to define the problem in terms of functions, which might notbe possible, thus restricting the ambit of application.

– A wide variety of functions is offered such as sine, cosine, tangent, logarithm,etc. This is undoubtedly an advantage but it is valid to wonder about theapplicability and utility with regards to the needs of the general public.

https://wwwnimbus.it.jyu.fi


– The technology in use significantly restricts interactivity, locking interaction toa logic of filling forms and clicking on Submit buttons.

– Version 4.1.1 was released on June 2007, whereas Version 4.1.2 was released onJuly 2013. This poses doubts about the active maintenance of the project.

– There is a version for commercial use called IND-NIMBUS.

Figure 3.8: Example of a screenshot from WWW-NIMBUS

3.2 Commercial Solutions

Table 3.2 provides a summary of the key features of commer-cial Multi-Criteria Decision Aiding solutions that are mentioned in[EURO Working Group on Multicriteria Decision Aiding ] or are otherwise very promi-nent in the market. Applications with invalid URL links have been excluded from analysis.

Even though this does not intend to be an exhaustive enumeration with respect toall the commercial solutions currently available, it is still a representative sample thatallows us to draw some general ideas about what the market offers:

• We can see that licence costs tend to be significantly expensive. This couldbe interpreted as a decision to exclusively target a market of business end-usersbut it leaves behind a people who are interested in using MCDA for personalpurposes.

• In the same vein as most non-commercial solutions, some of the commercial oneshave not been maintained for a few years.

• Apart from a few exceptions, similarly to what happens in the non-commercialworld, monolingualism seems to be the general rule in the landscape of commercialsolutions.

• As far as methodology are concerned, we can see that 6 out of 9 of the reviewedcommercial applications only support a single MCDA method.

3.2. Commercial Solutions 31

• The applications that support more than one MCDA method do not offer anyvisible instance of method choice. On the contrary, they seem to use the dif-ferent methods for different stages of the process as if they were not mutuallyexclusive. For example, we have seen a case where AHP is used for eliciting criteriaweights, whereas other methods are used for the other steps of ranking calculations.For our development, we prefer to avoid this approach so we will only apply themost suitable method according to each decision making scenario —onlyone method at a time—.

Table 3.2: Commercial Software SolutionsName Cost Methods Lang. Latest release

1000Minds20,000 USDper year1

PAPRIKA EN N/A (website)

D-Sight11,000 USDper year2

PROMETHEE,AHP, MAUT

EN, FR,JA

N/A (website)

Expert ChoiceNot dis-closed

AHP EN N/A (website)

Hiview3 1,600 USD 3 MACBETH EN August 2011 (3.2.0.7)OnBalance Free Beta SMART EN January 2010 (3.1.294)Smart Deci-sions

795 USD /2,995 USD 4

AHP, MAUT,SMART

EN June 2015 (3.2.5)

SuperDecisions Free Beta ANP EN January 2013 (2.2.6)

V.I.S.A.1,995 USD /4,995 USD 5 SMART EN 8.0

VisualPROMETHEE

1,400 USD6 PROMETHEE,GAIA

14 lan-guages

September 2013 (1.4)

The following subsections will provide further details about each case.

3.2.1 1000Minds

• Developed by: 1000Minds Ltd. (New Zealand).

• Website: http://http://www.1000minds.com.

• Platform: Web application.


– 1000Minds implements the PAPRIKA method, which stands for “PotentiallyAll Pairwise RanKings of all possible Alternatives”. It consists of a number of

1Discounts are available for charities and small organisations.2Licence for 20 users: 11,000 USD (10,000 EUR).3Standard, single-user licence with 1-year support: £1050.4“Gold” version: 795 USD (£495). “Platinum” version: 2,995 USD (£1,900).5Basic: 1,995 USD. Professional: 4,995 USD.6Business Edition Licence: 1,400 USD (1,250 EUR)

http://http://www.1000minds.com


questions and each one of them requires the decision maker to choose betweentwo hypothetical alternatives. Those alternatives are described according totwo criteria at a time from the set of criteria of the decision and a trade-offbetween those two criteria is always involved.

– This method is patented by 1000Minds Ltd. in the United States of America,New Zealand and Australia. The patent gives the company the right to forbidothers from implementing the method.

– The tool allows group decision-making.

– There are template decisions available for a wide variety of areas, including:Health technology and Patient prioritisation, HR Management, Project Man-agement, Investment, Social Services, Vendor Selection and Competition Judg-ing, among others.

Figure 3.9: Example of a screenshot from 1000Minds

3.2.2 D-Sight

• Developed by: D-Sight (Belgium).

• Website: http://www.d-sight.com.

• Platform: Web.


– In the past, the Desktop version of D-Sight used to implement PROMETHEEand GAIA methods, whereas the web version now reportedly featuresPROMETHEE, AHP and MAUT.

– However, the use of the different methods is not made explicit to the end userand it does not seem to be mutually exclusive either. AHP is visibly appliedduring the step of weights elicitation. In turn, PROMETHEE and MAUT seemto be applied for the other steps of results calculation.

– It is possible to invite multiple users to take part in the decision.

– Sensitivity Analysis allows experimenting with both weights and evaluations.

http://www.d-sight.com


Figure 3.10: Example of a screenshot from D-Sight

3.2.3 Expert Choice

• Developed by: Expert Choice Inc. (USA)

• Website: http://expertchoice.com.

• Platform: Web application (Developed with Apache Flex).


– The application has a nice visual aspect.

– It allows many features related to collaborative decision-making.

– There are several keyboard shortcuts to facilitate interaction.

Figure 3.11: Example of a screenshot from Expert Choice

3.2.4 Hiview3

• Developed by: Catalyze Ltd. (United Kingdom)

• Website: http://www.catalyze.co.uk/index.php/software/hiview3.

• Platform: Windows stand-alone application.


http://expertchoice.com

http://www.catalyze.co.uk/index.php/software/hiview3


– The application has not received any maintenance for more than 4 years, whichposes some doubts about its active status.

– The graphical interface might be considered as not modern but, most impor-tantly, it heavily relies on the menu toolbar and on the quick launch icons. Thismight be difficult to follow for certain groups of users, since the sequence oftasks to be performed is not clearly signalled.

– The company claims that the solution is equally suited for individual and groupdecision-making processes just because the screen is designed to be projected,which is arguable, since any screen may be projected. Letting that apart, theredoes not seem to be any facilitation for group tasks as seen on other decision-making applications.

Figure 3.12: Example of a screenshot from Hiview3

3.2.5 OnBalance

• Developed by: Quartzstar Software Ltd. (United Kingdom).

• Website: http://www.quartzstar.com/software.html.

• Platform: Windows stand-along application.


– At variance with other applications, OnBalance provides users with a so-calledProcess Diagram which depicts a high-level picture of the processes to be per-formed.

– Both Sensitivity and Robustness Analyses are available.

http://www.quartzstar.com/software.html


Figure 3.13: Example of a screenshot from OnBalance

3.2.6 Smart Decisions

• Developed by: Cogentus Consulting Ltd

• Website: http://www.cogentus.co.uk/products.



– There is a hyperlinks-based help file available but it is not integrated with theuser interface as on-screen help.

– The list of available methods was obtained from [Cogentus Consulting Ltd ].However, the application does not provide transparency with regards to themethods applied on each case.

– In particular, the application does not provide any method choice instancebut, on the contrary, it apparently tries to integrate the different methods.For instance, at the moment of defining criteria weights, the user may turnto pairwise comparisons (as in AHP but significantly different as it will beexplained further ahead) or Swing weighting (in a style consistent with Multi-Attribute Theory).

– As far as pairwise comparisons are concerned, the documentation mentionsAHP’s traditional evaluations (1, 3, 5, 7, 9) and links some verbal descriptionsto those values (as AHP also does), but the user interface does not providethose verbal descriptions for guidance neither does it restrict input values to 1to 9 range.

– Version 3.2.5 was released in June 2015 but the previous version, 3.2 is fromDecember 2013.

http://www.cogentus.co.uk/products


Figure 3.14: Example of a screenshot from Smart Decisions

3.2.7 SuperDecisions

• Developed by: Creative Decisions Foundation (USA).

• Website: http://www.superdecisions.com.



– Keyboard shortcuts are available for evaluations.

– Inconsistency indexes are shown as the user moves forward through evaluations.

– The software is still a beta version and is free for educational and research use.

– The price that will be charged for the software when it is released for generaluse has not been determined yet.

– Several unhandled technical exceptions happened while interacting with theapplication.

Figure 3.15: Example of a screenshot from SuperDecisions

http://www.superdecisions.com


3.2.8 V.I.S.A.

• Developed by: SIMUL8 Corporation (USA/UK).

• Website: http://visadecisions.com.



– A multi-level hierarchy of criteria may be set, which allows a rich model to bebuilt.

– The user is allowed to interactively experiment with the results of adjustingparameters.

– The interface style dates back to the days of MS-DOS and Windows 3.1.

– The website does not seem to be well-maintained as download links do notwork. After contacting the company, a different link was obtained, which madeit possible to download version 8 of the application. However, as of today, thepage still mentions version 7 as the latest version.

Figure 3.16: Example of a screenshot from V.I.S.A.

3.2.9 Visual PROMETHEE

• Developed by: VPSolutions.

• Website: http://www.promethee-gaia.net/software.html.



http://visadecisions.com

http://www.promethee-gaia.net/software.html


– The application provides different graphical visualisations inherent toPromethee method (Ranking, Diamond, Network, Rainbow, etc.).

– All the application is based on a single screen and interaction is performedthrough menu options and quick access icons. This might be difficult to followfor a non-expert user.

– The latest version of the application was released two years ago, which posessome doubts about its active maintenance.

Figure 3.17: Example of a screenshot from Visual PROMETHEE

Chapter 4

ElectioVis: A New MCDAApplication

“Once you make a decision, the universeconspires to make it happen.”

Ralph Waldo Emerson

“ElectioVis” is the compound name that has been chosen for the software solution.On the one hand, “Electio” means “decision” in Latin. On the other hand, the suffix“Vis” is meant to highlight the importance of visualisation aspects in a decisionanalysis tool.

“ElectioVis” is the result of more than 4 months of full-time work. The back-endpart of the solution is made up by 10,973 lines of code, whereas the front-end partconsists of 10,230 lines of code. Altogether, this represents more than 21,203 linesof code, which are important from a qualitative viewpoint, as the solution deliversnovelty and innovation in different ways.

The following sections intend to summarise key aspects of this endeavour.

4.1 Remarkable Features of ElectioVis

4.1.1 MCDA Method Choice

The “Parliamentary” Approach to MCDA Method choice has been explained from atheoretical perspective in Section 2.8. Let us now see how it has been implemented froma user-interaction perspective.

After users have answered questions, methods are assessed and it is possible tographically see points in favour (in green) and against (in red) for each method. Theuser is expected to choose the method that simultaneously has no vetoes (i.e. pointsagainst) and the highest number of approvals (i.e. points in favour).

However, that might not be always the cause. As discussed before, requirementsmight sometimes collide with each other and it is important to highlight conflictingpoints in order to assist the user to find a compromise.

For example, in the case illustrated in Figure 4.1, the user has picked answers “1a”,“2a”, “3a” and “4a”. We can see that the first, the third and the fourth answers are

40 Chapter 4. ElectioVis: A New MCDA Application

consistent with AHP, but the answer to question 2 is not.

Figure 4.1: AHP Method Assessment

Therefore, the person can review the choice regarding compensation. If such a choiceturns out to be non-negotiable, neither AHP nor MAVT are feasible options. Then, theassessments for the other methods may be reviewed by clicking on the relevant items ofthe grid. Figure 4.2 shows the assessments for Electre III.

Figure 4.2: Initial Electre III Method Assessment

At that point, the person may agree that a numerical score is desirable but not manda-tory and notices that an ordinal ranking might eventually be fine (Question 1). Similarly,he or she might be willing to express his or her preferences through numeric and cate-gorical values instead of pairwise comparisons (Question 3). If those two items can benegotiated, then the vetos for ELECTRE III disappear as seen on Figure 4.2, and thatmethod might be chosen.

4.1. Remarkable Features of ElectioVis 41

Figure 4.3: Subsequent Electre III Method Assessment

More examples of the method choice step may be seen in A.1.1, A.2.1 and A.3.1.

4.1.2 Value Functions (MAVT)

Value functions translate each of the performance assessments to a standardisedscore that ranges from 0 to 100.

By default, the system suggests a proportional linear value function, where thebest performance receives a score of 100 and the worst performance receives a score of 0.

In an objective to be maximised (the more, the merrier), the best performance isnaturally represented by the highest level that has been seen among all the alternatives,whereas the worst performance will be linked to the lowest level seen among thealternatives. In an objective to be minimised, things are the other way round.

The user needs to verify, on a criterion-by-criterion basis, if the default shapeof the function truly represents his or her assessments. If it is not the case, Electio-Vis offers three ways to customise the value function for quantitative criteria: usingThresholds, using the Bisection Method and through Direct definition via mousegestures. For qualitative criteria, only Direct definition is relevant. These methods areexplained as follows.

4.1.2.1 Thresholds

Assigning a minimum level of satisfaction and/or a level of complete satisfac-tion to the original linear shape.

Theminimum level of satisfaction is the first performance that starts deliveringsome value for the decision maker. In a criterion that needs to be maximised (e.g.the speed of a car), any performance less or equal than that value is assigned a score of“0”. On the contrary, in a criterion that needs to be minimised (e.g. the price of a car),


performances larger or equal than that value are assigned a score of “0”.

The level of complete satisfaction is the first performance that represents yourmaximum aspiration (i.e. a score of “100”). In criteria requiring maximisation, anyperformance greater or equal to this value is assigned a score of “100”. In criteria requir-ing minimisation, any performance less or equal than that value is assigned a score of “100”.

A combination of both parameters allows obtaining a wide variety of linear shapes(e.g. “Usual”, “U-Shape”, “V-Shape” and “Linear” as defined in Promethee terminology).An example of linear shape customisation within ElectioVis may be seen in A.1.5.2.

4.1.2.2 Bisection Method

A practical example of the application of the Bisection Method within ElectioVismay be seen in A.1.5.5. The reader is encouraged to alternate between this explanationand the aforementioned example in order to fully understand the underlying idea.

The ultimate goal of the Bisection Method is to find three points where the decisionmaker is indifferent about the degree of satisfaction of a particular criterion. During thefirst iteration, a score of “50” will be assigned to the middle indifference point. Then, thesame will happen with the scores of “25” and “75”.

Each iteration of the method involves dividing an interval in subintervals and doingcomparisons between them. The first subinterval (i.e. “Subinterval A”) will always endexactly on the same point where the second subinterval starts (i.e. “Subinterval B ”). Wewill refer to such a point as “Assessment Point”. This is outlined in Figure 4.4.

Figure 4.4: The Bisection Method involves comparing two subintervals

For the first iteration of the method, the “Lower Bound ” will be the lowest per-formance that has been identified among all the alternatives for a given criterion.Similarly, the “Upper Bound ” will be the highest performance of the same criterion.In the first comparison, the “Assessment Point” will be exactly midway between the“Lower Bound ” and the “Upper Bound ”.

After considering both subintervals, the decision maker might well be indifferent atthat point, which would mean that we can conclude the first iteration of the methodby placing the score of “50” there. But sometimes, he or she might not be indifferentin the first comparison. Therefore, we need to adjust the “Assessment Point”, either bymoving it leftwards —if the user said that “Subinterval A” was preferred— or by moving


it rightwards —if “Subinterval B ” was preferred—.

After the adjustment, a new comparison takes place and this is repeated untilthe indifference point is finally found. At that moment, the score of “50” is placedand the first iteration concludes. This gives way to the second iteration, which appliesrecursion within the range determined by the “Lower Bound ” and the last “AssessmentPoint” to assign the score of “25”. Finally, the third iteration happens to assign the scoreof “75”, within the range determined by the “Assessment Point” and the “Upper Bound ”.

Whenever a decision-aiding analyst is involved, it is the professional who defines theimportance of the adjustments of the “Assessment Point” while indifference is beingsought. However, a software application needs to be able to cope with that automatically.ElectioVis implements a custom algorithm designed to automatically calculate thebreadth of the subintervals to be compared and depicts them in a graphical way. Theseare two distinctive features that have not been seen in other software solutions.

Basically, the algorithm has been designed with a number of goals in mind:

• Adjustments should be consistent with the distance between the “AssessmentPoint” and either the “Lower Bound ” –if moving leftwards– or the “Upper Bound ”–if moving rightwards–. Bigger distances should entail more dramatic adjustmentsand, conversely, smaller distances should imply more subtle adjustments.

• Furthermore, adjustments should not be too quick nor too slow. Besides, they shouldbe smoother as the pertinent bound gets closer after each comparison.

• As it will be shown in test cases, moving from one tenth to one fifth of the distancebetween the “Assessment Point” and the pertinent bound proved to be reasonableon a wide variety of cases.

• Whenever possible, adjustments should be performed in rounded figures in orderto make them more intuitive to the eyes of decision makers (e.g. multiple ofnumbers which may be calculated as powers of 10, their halves or their quarters).

We will first explain the steps in functional terms and then we will formalise it ina symbolic way in Algorithm 1. Let us assume that we are in the first iteration of the“Bisection Method ”. We start off by defining the two subintervals of the iteration:

• Subinterval A ranges from the “Lower Bound ” to the average between the “LowerBound ” and the “Upper Bound ” (this average is our initial “Assessment Point”).

• Subinterval B ranges from the “Assessment Point” to the “Upper Bound ”.

As noted before, if the user indicates that “Subinterval A” is preferred, we need tomove the “Assessment Point” towards the left and ask a new question. On the contrary,if the user indicates that he/she prefers “Subinterval B ”, we need to move the “AssessmentPoint” towards the right and ask a new question. If we face the first case, we proceed asfollows:


1. The difference between the “Lower Bound ” and the “Assessment Point” is calculatedand the result is divided by 10. This is our “Tentative Gap” but it might need to berounded.

2. We take Logarithm in base 10 of the previous result and round up to the next integer.Let us call this result δ.

3. Using the previous result, we get four “Candidate Rounded Gaps”: 10 to the powerof δ+1, the latter divided by 2, 10 to the power of δ and the latter divided by 2.

4. We compare the four “Candidate Rounded Gaps” with the “Tentative Gap” obtainedin step 2.

5. The smallest difference determines which of the four “Candidate Rounded Gaps”should be chosen. We call this “Rounded gap”.

6. We divide the difference between the “Lower Bound ” and the “Assessment Point”by the “Rounded Gap”, thus getting the number of chunks between both of them.

7. If the number of chunks is greater than 10, it means our gap might be too small.Therefore, we multiply the “Rounded Gap” by 2 until that stops happening. Thefinal product becomes the “Corrected Gap”. Otherwise, the “Rounded Gap” becomesthe “Corrected Gap” without any transformation.

8. However, if the new number of chunks is less than 5, it means that our gap mightbe too broad. That is the reason why, under those circumstances, we divide the“Corrected Gap” by 2 to obtain the “Final Gap”.

9. We subtract the “Final Gap” to the original “Assessment Point”, thus obtaining anew “Assessment Point”. We build two new intervals and ask the user again to seeif he is indifferent or prefers one of them. In the latter case, we start a new iterationof the algorithm.

The symbolic formalisation may be found in Algorithm 1. On the other hand, tableB.1 in Appendix B shows the range of tests that were performed to make sure that thegoals described in 4.1.2.2 were achieved.


Algorithm 1 Calculate the new “Assessment Point”Input lb: Lower bound, ub: Upper bound, ap: Assessment point1: tentativeGap ← (ap - lb) / 102: δ0 ← log10(tentativeGap)

3: δ ← min(n) | (n ∈ Z ∧ n ≥ δ0)

4: cand1 ← 10δ+1

5: cand2 ← cand1 / 2

6: cand3 ← 10δ

7: cand4 ← cand3 / 2

8: for n = 1 to 4 do9: difn ← |candn − tentativeGap|

10: minDif ← minn=1...4(difn)

11: for n = 1 to 4 do12: if minDif = difn then13: roundedGap← candn14: chunks← (ap− lb)/roundedGap15: while chunks > 10 do16: roundedGap← roundedGap ∗ 2

17: chunks← (ap− lb)/roundedGap18: correctedGap← roundedGap

19: if chunks < 5 then20: finalGap← correctedGap/2

21: else22: finalGap← correctedGap

23: return ap - finalGap

4.1.2.3 Direct Input

It is also possible to adjust existing points in the value function chart through drag-and-drop mouse gestures. It is also possible to add new points by pointing and clickingon the chart too.

4.1.3 Swing Weights Elicitation (MAVT)

This is another distinctive feature that cannot be seen in other MCDA pieces ofsoftware. A complete case of Swing Weights elicitation within ElectioVis may be foundin A.1.6.

As seen on Figure 4.5, the Swing Weights screen in ElectioVis compares best andworst alternatives —highlighted in green and red, respectively— as far as each criterionis concerned.


Figure 4.5: Swing Weights Screen in ElectioVis

The decision maker needs to choose the criterion that, from his or her perspective,presents the most important contrast and assign a score to it. Normally, the firstone receives a score of “100”. Then, the second most important contrast needs to bechosen, a score is assigned and so it goes on until all criteria are scored. Each scoreneeds to be less than or equal than the previous one.

The screen allows disabling charts related to criteria that have already been scored.This is meant to make it possible for the decision maker to focus on the remaining criteria.During the whole process, normalised Swing Weights may be seen on the bar chartsituated on the right hand side of the screen.

4.1.4 Revised White Cards Approach (ELECTRE III)

From the set of other software solutions that were surveyed as part of this project,those implementing ELECTRE require the direct input of weights. At variance withthat, ElectioVis presents a weight elicitation mechanism that is based on Simo’s revisedapproach, presented in [Roy 2002].

Figure 4.6 shows an example of the Weight Elicitation screen in ElectioVis. A completecase and the underlying calculations may be found in subsection A.2.6.

Figure 4.6: Example of Weights Elicitation in ElectioVis through White Cards


4.1.5 Monte Carlo Methods (AHP and MAVT)

Monte Carlo Methods are a broad class of computational algorithms that rely on re-peated random sampling to obtain numerical results. They are pretty well-known in theProject Management discipline —as described in [Project Management Institute 2013]—to play what-if scenarios with the purpose of calculating the probability of delivering aproject on time and within budget.

In the context of MCDA discipline, we can also take advantage of these methods.In particular, ElectioVis uses this approach to test Robustness of the decision-makingmodel as far as criteria weights are concerned and, in the future, it will also be used forSensitivity Analysis of weights —preserving the original ordinal rankings defined byusers— and performance assessments.

The robustness assessment currently implemented in ElectioVis runs 1,000 randomtests, which consist in automatically assigning a random weight from “0” to “100”to each of the criteria and recalculating rankings on the basis of them. Then, a chartis shown to indicate how frequently each alternative was ranked as the best, as thesecond best, as the third, etc. That helps the decision maker notice if the originalresults are consistent with these trends or if they are unstable.

The use of Monte Carlo Methods in MCDA may be seen in other software solutionsas well, but it should be noted that, so far, it has not been a generally available feature.Figure 4.7 shows an example of Monte Carlo Robustness Assessment in ElectioVis.

Figure 4.7: Example of Monte Carlo Robustness Assessment in ElectioVis

4.1.6 Contextual Help

On-Screen contextual help is quite a difficult feature to find in other software solutions.Normally, if help exists, it is provided as a separate document or help file with hyperlinks.


ElectioVis intends to provide timely and accessible help all over the decision-makingprocess. As it may be seen in Figure 4.8, this help is provided through pop-up windows.

Some of the pop-ups are initially shown by default but they may be disabled ifusers do not want to see them. Other pop-ups are to be activated by the users when, forinstance, they would like to get additional help on how to fill in a particular column ashighlighted in Figure 4.8.

Figure 4.8: Example of Contextual Help in ElectioVis

4.2 Architecture

Figure 4.9 shows the diagram of the general architecture of ElectioVis.

Figure 4.9: ElectioVis Architecture

Chapter 5

Entrepreneurship, Legal andAdministrative Aspects

5.1 Logo

Logos are cornerstones for developing new brands. Due to the importance of thetask, a graphic designer was hired to draw up the logo of ElectioVis.

After completing a template document which gathered information about thecompany, the product and visual preferences —among other aspects—, a proposalwith 11 candidate logos was received, which included 6 main models in colour and 5alternative versions in back and white.

One of the models in black and white was chosen and we collaborated with each otherto define colours. The final version of the logo may be seen in Figure 5.1. The light bluepart is inspired on the flag of the Argentine Province of Tierra del Fuego, whose capital,Ushuaia, is the southernmost city of the World —and a natural marvel—.

Figure 5.1: Product Logo

The iconic part of logo has a number of symbolic meanings, as highlighted in Figure5.2. Some of them are more evident than others.

Figure 5.2: Symbolic Parts of the Logo

To start with, the grey part that is in the centre of the icon has the shape of ahuman figure. This shape symbolises people from the general public, who are the

50 Chapter 5. Entrepreneurship, Legal and Administrative Aspects

main protagonists in the design of the solution.

If we consider the green part of the icon in conjunction with the grey circle, we canthink of a torch or lantern that is illuminating a path. In turn, if we consider only thelowest green stroke in conjunction with the grey circle, we can perceive a magnifyingglass as well. Both of them are linked to discovering information that is not evidentduring alternatives comparison.

If the icon is considered as a whole, we could see something that resembles an eye aswell. The eye is connected to vision and vision is connected to visualisation, another ofthe key aspects of the solution.

Finally, if we consider the blue strokes in addition to the grey circle, we could think ofa milestone from where different paths bifurcate. This naturally represents mutuallyexclusive alternative.

5.2 Company Registration in France

Given the list of legal forms admitted under French law —whose quick comparisonmay be found in [L’Agence pour la Création d’Entreprises 2015]—, the final choice was“Société par Actions Simplifiée Unipersonnelle” (SASU ) which, as its nameindicates, is a simplified form of private company limited by shares.

There are only three forms that accept a single associate and they are SASU, EURL—“Entreprise Unipersonnelle à Responsabilité Limitée”— and “Entreprise Individuelle”.At variance with the other two cases, a SASU may be easily transformed into a SAS toaccept multiple associates, as long as statutes have provisions that accommodate to in-dividual and plural government. That was a compelling advantage that defined the choice.

On the other hand, it was decided to name the company as “Idées du Sud SASU ”.“Idées du Sud” may be translated as “Southern Ideas” or “Ideas from the South”. There arethree symbolic meanings linked the name: the company has its legal siege in Châtenay-Malabry —located to the South of Paris—, I was born in Argentina and my grandparentswere born in the province of Cosenza —in the South of Italy—.

5.2.1 Public Registry of Commerce and Companies

In order to register a company in France, the following documents are required:

• “Statuts”: This document describes all the rules that govern the legal entity beingcreated. In this case, it was a 7-page document in French that was developed on thebasis of three different sources available online.

• “Attestation de Parution”: A publication has to be done in a regional newspa-per, informing about the creation of the company, its address, its management, the

5.3. Brand Registration 51

amount of social capital, an so forth. After performing this, the pertinent newspaperissues a certificate that gives faith of the publication.

• A self-certified copy of the passport. Non-EU citizens also need an authorisationfrom the Prefecture.

• “Déclaration de Non-Condamnation”: It is an affidavit where the person swearsnot having been sentenced by justice.

• “Depôt de capital social” and “Liste de Souscripteurs d’Actions”: Thesedocuments are issued by a bank, after a current account is opened on behalf of thesociety and the social capital is deposited on it.

• “Certificat de Domiciliation”: This document proves having the right to usethe address that is being declared as the siege of the company. In my case, ECPIncubator kindly let me use their address for the declarations.

I highly encourage any people who might be interested in following the same process togo in person to the relevant office of the “Greffe” (located in Nanterre for the Departmentof Hauts-de-Seine) and take the physical documents (in paper). I made the mistake oftrying to use the internet service “Guichet Entreprises”, which complicated and help upthe process considerably.

5.2.2 Revenue Collection Service

A copy of the statutes needs to be deposited in the offices of the national revenuecollection service. In return, they seal other copies acknowledging the fulfilment of theduty.

5.3 Brand Registration

On April 22, 2015, a trade mark registration request has been filed with the Officefor Harmonisation in the Internal Market.

Once the petition is accepted, this will give protection to the name and logo of Elec-tioVis over the entire territory of the European Union. As seen on Figure 5.3, the periodfor posting oppositions finished by July 29 so the next steps are merely administrative.

Figure 5.3: ElectioVis Brand Registration

52 Chapter 5. Entrepreneurship, Legal and Administrative Aspects

5.4 Database Registration

As a French organisation, Idées du Sud SASU is bound by Act 78-17 of 6 January1978 on Information Technology, Data Files and Civil Liberties.

The law requires the registration of any database containing information that coulddirectly or indirectly identify people. Even though no sensitive data is stored inthe database of ElectioVis, as names, surnames and IP addresses are stores, therequirement is in force. The obligation was fulfilled through the website of the “Commissionnationale de l’informatique et des libertés” (CNIL).

Chapter 6

Usability Testing and FeedbackReceived

“We have two ears and one mouth so thatwe can listen twice as much as we speak”

Epictetus

6.1 Registered Users

By the end of July 2015, a total number of 92 users had registered in ElectioVis. Itshould be noted that the sample was influenced by the author’s network of contacts.

In particular, as Figure 6.1 shows, 21 out of those 92 users (22.83 %) were partof the general public. As for the rest, some were met by the author either at University ofBuenos Aires or during the participation in the Erasmus Mundus IT4BI Programme (bothincluded under the category “University” in the chart). The sample also includes formerworkmates, contacts from INRIA or the PMI Chapter France (“Contacts in France”),friends from High School and only a few members of the family.

Figure 6.1: Registered Users by Type of Connection

As far as age is concerned, as depicted in Figure 6.2, we can see that 6 out of 10registered users are from 20 to 34 years old.

54 Chapter 6. Usability Testing and Feedback Received

Figure 6.2: Registered Users by Age and Gender

From a generational perspective, more than a half of registered users belong in“Generation X ” group (people born between 1965 and 1984), whereas more than 3 outof 10 belong in “Generation Y ” group (people born beeen 1985 and 2000). This is shownin Figure 6.3.

Figure 6.3: Registered Users by Generation and Gender

This “generational bias” might be partly explained by the channels used to reachusers but it is also true that the connection to technology is not even across differentgenerations. At any rate, the generational aspect needs to be taken into account at themoment of defining the next steps of the project, since different generations evidenceparticular preferences and needs.

6.2 Decisions

At the moment of analysing interactions with the system, we can see that, in total 63decisions were created. This excludes items created and subsequently deleted by users

6.2. Decisions 55

because, even though the system allows traceability through logical deletions, in mostcases, deleted items turned out to be simple tests.

Major topics for the created decisions included, among others:

• Holidays (11 cases):

– Choosing a destination.– Picking a hotel.– Selecting an airline.

• Deciding on acquisitions (9 cases):

– Cars (6 cases).– Musical instruments.– Tablets.– Mattresses.

• Choosing a place to live (6 cases):

– Apartment.– Neighbourhood.– City.– Country.

• House-related (4 cases):

– Maintenance.– Decoration.– Picking a Cable company.– Buying or Adopting a Pet.

• Financial and Investment Decisions (2 cases):

– Getting a credit.– Buying real state.

• Leisure activities (2 cases):

– Theatre/cinema.– Restaurants.

• Work-related (2 cases).

• Personal Decisions(2 cases):

– Deciding political vote– Choosing a boyfriend... (indeed).

• Academic

– Should I enrol in summer courses?


6.3 Method Choice

As for Method Choice, Figure 6.4 that MAVT has been the most chosen frequentlymethod, followed distantly by AHP and ELECTRE III.

Figure 6.4: MCDA Methods Chosen

To get a more detailed perspective, let us now analyse how users answered methodchoice questions. In the first question of the Method Choice step, as seen on Figure 6.5,there were 33 out of 63 cases where numeric results were preferred. That can onlybe achieved through MAVT and AHP. In 3 of them, people still chose ELECTRE IIIwithout paying attention to the veto.

Figure 6.5: MCDA Method Choice — Question 1

On the other hand, in other 21 cases out of the 63 total decisions, people replied

6.3. Method Choice 57

that were fine with an ordinal ranking, which may be obtained through any of thethree methods. Finally, categories were only preferred by a small minority. The threemethods can also deal with that scenario.

Question 2 refers to compensations among criteria. In the original versionof the questionnaire, this item was not being discriminative enough. As itwas noted before, there used to be a neutral answer to allow cases where compen-sations were desirable but not mandatory (answer “2b”). In turn, answer “2a” meanta rejection to compensation and answer “2c” meant that compensation was a requirement.

As seen in 6.6, answer “2b” was being chosen by the majority but this poses doubtsas to whether people were simply choosing it because they were not being able tounderstand the concept of compensation. Therefore, it has been decided to increasecontextual help for this question and remove the neutral answer to force a choicebetween compensation or lack of compensation.

Again, it is surprising to see that, 8 out of 14 people who rejected compensation stillchose AHP or MAVT. This suggests that either double confirmation should be requiredor vetoes should directly forbid users from continuing unless answers are changed.


Question 3 shows that preference elicitation choices were pretty much evenly dis-tributed. Answers “3b” and “3c” are compatible with either MAVT or ELECTRE III.On the other hand, answer “3a” is only compatible with AHP but still, in 9 cases, peopledid not pay attention to vetoes and chose MAVT or ELECTRE III as shown in Figure6.7.



The answers to Question 4, depicted in 6.8, show that an overwhelming majority seemsto be indifferent to possible rank reversals.


However, we consider that it is better to redesign the question in order to focus onstability of alternatives instead:

4. How stable is the list of options from where you are choosing?

(a) Once I define them, I am sure they will not change.

(b) They might change but, if they do, I will create a new decision.

(c) They might change and, if they do, I want to keep consistency without creatinga new decision.

6.4. Survey Questions 59

Answers “4a” and “4b” would equally reward the three alternatives because the threeof them are suitable in case independence to third alternatives is not required.

Answer “4c” would reward MAVT but would penalise AHP and ELECTRE III.

6.4 Survey Questions

While users are logged in to ElectioVis, a link to a survey is shown in the upper partof the screen. This is illustrated in Figure 6.9.

Figure 6.9: ElectioVis Survey Link

The main purpose of the survey is to obtain feedback about opportunities ofimprovement of the software solution.

The survey covers four main areas: Interface, User Experience, Help and OverallOpinion. All areas include three or four rating questions —which request a score from“0 ” to “10 ”— as well as an open question where the user is encouraged to freely commentor make improvement suggestions.

Interface questions cover the perceived ease of use, efficiency of the interaction, thevisual aspect of screens and the clarity of results visualisations. These questions may beseen on Figure 6.10.

User Experience questions ask about the speed of response of the application, theopinion about the chance to visualise results interactively and the logic of the steps to befollowed. These questions may be seen on Figure 6.11.

Help questions ask about usefulness, conciseness, clarity and understandability ofon-screen instructions that are provided through pop-up windows. These questions maybe seen on Figure 6.12.

Finally, Overall questions deal with the general opinion of the user. Besides, theyask if the end-user thinks that ElectioVis helped him or her to make a better decisionand they also check if the person is planning to user ElectioVis again in the future and/orrecommend it to other people. These questions may be seen on Figure 6.13.


Figure 6.10: ElectioVis Survey — Questions about Interface

Figure 6.11: ElectioVis Survey — Questions about User Experience

Figure 6.12: ElectioVis Survey — Questions about Help

6.5. Survey Answers 61

Figure 6.13: ElectioVis Survey — Overall Questions

6.5 Survey Answers

In total, 31 people, which represent around one third of registered users, com-pleted the survey. Figures 6.14 and 6.15 show age groups and generation composition.

Figure 6.14: Survey Respondents by Age and Gender

Figure 6.15: Survey Respondents by Generation and Gender


We will discuss answers to quantitative and qualitative questions. To start with,Figure 6.16 shows the average scores assigned by users in quantitative questions, groupedby age ranges. Scores that are less than 7 appear highlighted in yellow.

Figure 6.16: Average Scores by Age Group

At first sight, it seems that Interface and On-Screen Help might pose opportunities ofimprovement. Let us now get into the details of answers for each group.

6.5.1 Interface

Figure 6.17 shows the number of times each score —from “0 ” to “10 ”— was assignedwith regards to Interface questions.

Figure 6.17: Answers about Interface

Main ideas expressed in the qualitative question are enumerated as follows:

• It would be helpful to have templates or examples created on new accounts (8 cases).

• Videos or audio tutorials could also help (6 cases).


• Comments and user actions evidence that Categories might require additional clarity(4 cases).

• Fonts in Help Popoups and Results screen might need being reviewed (2 cases).

• It would be nice to highlight completed comparisons (AHP).

• The interface is intuitive, easy-to-use and aesthetics are consistent with standard ap-plications. However, nowadays, the visual appeal of new web applications is settingother levels of expectations.

• One user suggests adding more colours to make screen more appealing.

• Another user says that the interface is not visually friendly.

• Results screen had problems of visibility on some monitors (this has been alreadyfixed).

• One user suggested adding an explicit ranked list of alternatives, in addition tocharts currently available.

6.5.2 User Experience

Figure 6.18 shows assigned scores with regards to User Experience questions.

Figure 6.18: Answers about User Experience


• Some users suggest automatically saving changes. Others suggest a confirmationmessage with “Save” and “Discard ” options (5 cases).

• Criteria, Alternatives and Performance are crucial steps. It is important to makesure that they are correctly input so that subsequent screens can provide meaningfulfeedback, otherwise it might be easy to lose confidence on the application.

• It has been suggested to define a more strict step-by-step procedure, without possi-bility to move forward without completing previous steps.

• It should be possible to input alternatives first.


• Passwords should be freely determined by users.

• Names of key concepts should be more intuitive.

6.5.3 On-Screen Help

Figure 6.19 shows assigned scores with regards to On-Screen Help questions.

Figure 6.19: Answers about On-Screen Help


• Explanations should be simpler or less technical (4 cases).

• Help should be more interactive and available upon request (3 cases).

• Help should be summarised (2 cases).

• Maybe explanations should be shorter, with an option to “View more”.

• Help should be integrated with the screen instead of showing it as pop-ups.

• Graphical explanations could help people with less technical backgrounds.

• Some aspects have redundant explanations. Once a user has acknowledged a partic-ular help message, it should not appear again for new decisions.

6.5.4 Overall Opinion

Figure 6.20 shows assigned scores with regards to Overall Opinion questions.

Figure 6.20: Answers about Overall Opinion



• Several comments summarise what has already been discussed in previous sections.

• Sharing to Social media and Exporting to PDF could be nice features.

• Compatibility with mobile devices would be advisable.

Chapter 7

Conclusions and Perspectives

7.1 Lessons Learnt

7.1.1 User Guidance

On-Screen Help is an area with a perceived potential of improvement. Making itconcise and accessible is of utmost importance to achieve the objective of democratisingdecision aiding among non-technical audiences.

Some steps have already been taken in that direction such as decentralising helpthrough help icons —which make help available upon request— and highlighting keyaspects in with bold font, underlining and colour. However, there is still a long way to go.

The use of multimedia tutorials as well as the availability of templates and illus-tration examples will be good complements to on-screen help, as already pointed outby an important number of users.

7.1.2 User Experience

Some simple features such as being able to change passwords or saving changesautomatically were not considered for the first version as effort had to be focused on awide variety of other aspects. However, as it was evidenced in the collected feedback, itseems that they make quite a difference for users.

Nowadays, users inevitably compare application interfaces with top-notch web-sites. It could be argued that some comparisons might not be relevant at all as companieslike Google or Facebook are in a very different position as far as resources are concerned.The epigraph of the Chapter 1 in [Ariely 2010] expresses this in a very compelling way:“Why Everything Is Relative — Even When It Shouldn’t Be”. In Ariely’s words, relativecomparisons are part of our predictable irrationality, so we have to accept and play by thethose rules.

7.1.3 Interface

From a front-end perspective, Apache Flex turned out to be a good starting point forElectioVis. As it was an already known technology for the author, it made it possibleto deliver an ambitious project on time and within planned scope.

That being said, Apache Flex presents a number of disadvantages when it comes tocompatibility with mobile devices and visibility to search engines. It is also

68 Chapter 7. Conclusions and Perspectives

becoming to be perceived as old-fashioned by some groups of highly demanding users.

Those are the main reasons why it should be analysed if the front-end technologyis to be kept or if a migration might be advisable for the sake of excellence.

7.2 Conclusion

As we have seen during Usability Testing, independently from which methods wereeventually chosen, answers to method choice questions evidence a remarkable diversityof decision-making contexts.

In order to be able to cope with such a diversity, a heterogeneous set of methodsis required. The “Parliamentary” Approach to MCDA Method Choice is an importantcontribution towards enabling multiple methods in a single tool. The list of questionsis expected to evolve in the future but the grounds for a significant break-troughhave been undoubtedly settled.

This academic contribution has been delivered in conjunction with the developmentof a software solution of important magnitude. ElectioVis is the first open-sourceMCDA solution that intends to deliver the needs of the general public. It provides helpand guidance not only during the MCDA Method Choice step but also during the entireprocess. It also applies different approaches in eclectic ways and it delivers novelty in anumber of user-interaction aspects such as Swing Weights Elicitation, Bisection Methodand White cards Weights Elicitation revised procedure among others.

As any human creation, ElectioVis is far from being perfect but it is perfectible ifefforts keep on being applied.

7.3 Future of the Project

In the context of a competition organised by the French Institute for Research inComputer Science and Automation1, ElectioVis has been awarded with the “Special Prizeof the Jury”. This prize involves the opportunity to sign a one-year contract with INRIAin order to continue the development.

Dimensions for future work include, but are not limited to: Implementing other MCDAmethods, enabling collaborative decision-making, continuous improvement of on-screenhelp and general user experience as well as review and potential migration of front-endtechnology, among others.

1Institut National de Recherche en Informatique et en Automatique (INRIA)

Appendix A

Illustration Cases

A.1 Multi-Attribute Value Theory

A.1.1 Decision Context

Let us imagine that a decision maker wants to buy a car and that his decision-makingscenario has the following features:

• He would like to obtain a numerical score for each of the cars he is analysing.

• If a car is very good in one aspect but very bad in another aspect, he would likethose contradictory performances to be compensated.

• He would like to compare cars using numbers for some objectives and categoriesfor some other objectives.

• He would like to keep the consistency of assessments in case more potential carsare later added after completing one process of analysis (also known as “independenceto third alternatives”).

Having to choose between Multi-Attribute Value Theory, Analytic Hierarchy Processand ELECTRE III, the first of them, i.e. Multi-Attribute Value Theory, seems to bethe most suitable method for this decision-making scenario.

In the Method Choice screen, we can see that ElectioVis recognises Multi-AttributeValue Theory as the only method with no vetoes and the highest number of approvals, asseen on Figure A.1.

Figure A.1: Method Choice in ElectioVis

70 Appendix A. Illustration Cases

A.1.2 Objectives

Furthermore, let us assume that the decision maker is interested in:

• Maximising the Speed of the car (measured quantitatively in km/h).

• Minimising the Price of the car (measured quantitatively in Euro).

• Maximising the degree of Comfort of the car (measured quantitatively as a scorefrom 1 to 5).

• Maximising the degree of Security of the car (measured qualitatively by means ofone of the following categories: “Very unsafe”, “Unsafe”, “Normal”, “Safe” or “Verysafe”).

• Minimising the periodic Maintenance Cost of the car (measured quantitatively inEuro).

• Maximising the appeal of the Colour of the car (measured qualitatively by meansof one of the following categories: “Ugly”, “Normal” or “Nice”).

Figure A.2 shows how the information about the measuring attributes is entered inElectioVis.

Figure A.2: Objectives in ElectioVis

A.1.3 Alternatives

We will assume that the decision maker wants to choose a car among a group of fouralternatives:

• Car 1

• Car 2

• Car 3

• Car 4

A.1. Multi-Attribute Value Theory 71

Figure A.3 shows how alternatives were input into ElectioVis.

Figure A.3: Alternatives in ElectioVis

A.1.4 Performance Assessments

The next step involves gathering the values of the attributes defined in the Objectivesscreen for each of the alternatives.

Table A.1 summarises the values that will be used for this example:

Table A.1: Performance AssessmentsSpeed Price Comfort Security Maintenance Colour

Car 1 247 29,000 5 Normal 200 UglyCar 2 216 17,000 4 Safe 100 NiceCar 3 180 21,500 3 Unsafe 150 NormalCar 4 205 9,700 0 Very unsafe 50 Nice

Figure A.4 shows how the information about performance assessments is entered inElectioVis.

Figure A.4: Performance Assessments


A.1.5 Value Functions

A.1.5.1 Speed

For “Speed”, let us assume that the user accepts the default proportional linear shapewith a value of 0 for the worst performance (180 km/h) and a value of 100 for the bestperformance (247 km/h).

Figure A.5 shows how the function is displayed in ElectioVis.

Figure A.5: Value Function for “Speed” in ElectioVis

Standardised scores may now be calculated as shown in Table A.2.

Table A.2: Standardised Scores for Speed

Original Value Standardised Score

Car 1 247 100

Car 2 216216− 180

247− 180* 100 = 53.73

Car 3 180 0

Car 4 205205− 180

247− 180* 100 = 37.31

A.1.5.2 Price

Similarly, the system suggests again a proportional linear shape for “Price”. In thiscase, it is an objective to be minimised so the value function is descending. The worstperformance (29,000 EUR) has a value of 0 and the best performance (9,700 EUR) has


a value of 100.

But let us assume that the user is not totally satisfied with the default shape andtherefore wants to customise it. He might, for instance, be totally satisfied with any priceof 12,000 EUR at most (level of complete satisfaction) and openly reject any price thatis higher than 20,000 EUR (minimum level of satisfaction). Bear in mind that valuesare inverted in this case because it is a minimisation objective. That explains why theminimum level of satisfaction is in the range of the highest levels.

Figure A.6 shows how this customised function is displayed in ElectioVis.

Figure A.6: Customised Value Function for “Price” in ElectioVis


Table A.3: Standardised Scores for Price


Car 1 29,000 0

Car 2 17,000 100 -17, 000− 12, 000

20, 000− 12, 000* 100 = 37.50

Car 3 21,500 0

Car 4 9,700 100

A.1.5.3 Comfort

For “Comfort”, let us assume that the user accepts the default proportional linearshape with a value of 0 for the worst performance (score 0 ) and a value of 100 for the


best performance (score 5 ).

Figure A.7 shows how the function is displayed in ElectioVis.

Figure A.7: Value Function for “Comfort” in ElectioVis


Table A.4: Standardised Scores for Comfort


Car 1 5 100

Car 2 33− 0

5− 0* 100 = 60

Car 3 44− 0

5− 0* 100 = 80

Car 4 0 0

A.1.5.4 Security

For “Security” the default proportional linear shape assigns a value of 0 to theworst performance (“Very unsafe”) and a value of 100 to the best performance (“Verysafe”). In turn, “Unsafe”, “Normal” and “Safe” receive scores of 25, 50 and 75 respectively.

Figure A.8 shows how this default function is displayed in ElectioVis.


Figure A.8: Original Value Function for “Security” in ElectioVis

Let us assume that the user wants to perform a small customisation as far asperformance “Safe” is concerned. He wants to assign a value of 100 instead of 75, thuslevelling up “Safe” to the same score of “Very unsafe”. This may be done by pointing andclicking on the chart at the desired level.

Figure A.9 shows how this customised function is displayed in ElectioVis.

Figure A.9: Customised Value Function for “Security” in ElectioVis



Table A.5: Standardised Scores for Security


Car 1 Normal 50

Car 2 Safe 100

Car 3 Unsafe 25

Car 4 Very unsafe 0

A.1.5.5 Maintenance Costs

For “Maintenance Costs” the default proportional linear shape assigns a value of 0 tothe worst performance (200 EUR) and a value of 100 to the best performance (50 EUR).Again, this is a minimisation objective so the function is descending.


Figure A.10: Original Value Function for “Maintenance Cost” in ElectioVis

It will now be illustrated how this value function may be customised by using theBisection Method. Figure A.11 shows the first question of the Bisection Method in Elec-tioVis. There, the user is asked to compare the intervals [50 EUR, 125 EUR] and [125EUR, 200 EUR] in order to determine either preference or indifference.


Figure A.11: First Bisection Method Question for eliciting “Maintenance Cost” ValueFunction in ElectioVis

If the user indicates that he prefers the first interval, the boundary between bothintervals (i.e. 125 EUR) is slightly moved to the left towards 115 EUR. Then, a newquestion is raised, asking to compare intervals [50 EUR, 115 EUR] and [115 EUR, 200EUR], as seen on Figure A.12.

Figure A.12: Second Bisection Method Question for eliciting “Maintenance Cost” ValueFunction in ElectioVis

If the user still prefers the first interval, the boundary is again slightly moved to theleft from 115 EUR to 105 EUR and a new question is raised as seen on Figure A.13.


Figure A.13: Third Bisection Method Question for eliciting “Maintenance Cost” ValueFunction in ElectioVis

Once more, if the user still prefers the first interval, the new boundary will be 95 EURand a new question will be raised, asking the comparison between the intervals [50 EUR,95 EUR] and [95 EUR, 200 EUR], as seen on Figure A.14.

Figure A.14: Fourth Bisection Method Question for eliciting “Maintenance Cost” ValueFunction in ElectioVis

Let us now imagine that the user is indifferent between the intervals [50 EUR, 95EUR] and [95 EUR, 200 EUR]. If that is the case, the point 95 EUR will be assigned avalue of 50 and this will be reflected in the chart of the function.

Moreover, the intervals [50 EUR, 95 EUR] and [95 EUR, 200 EUR] need to be fur-ther divided into subintervals, in order to find two new indifference points, which will beassigned the values 75 and 25 respectively. This is done in two steps, and the first one,


involving the comparison of subintervals [50 EUR, 72.5 EUR] and [72.5 EUR, 95 EUR],may be seen on Figure A.15.

Figure A.15: Fifth Bisection Method Question for eliciting “Maintenance Cost” ValueFunction in ElectioVis

For simplicity’s sake, let us assume that the user is indifferent between subintervals [50EUR, 72.5 EUR] and [72.5 EUR, 95 EUR]. Therefore, 72.5 EUR is assigned a value of 75.

On the next step, the interval [95 EUR, 200 EUR] is further divided into two subinter-vals: [95 EUR, 147.5 EUR] and [147.5 EUR, 200 EUR], which the user needs to compare,as seen on Figure A.16.

Figure A.16: Sixth Bisection Method Question for eliciting “Maintenance Cost” ValueFunction in ElectioVis

If the user is indifferent again, 147.5 EUR is assigned a value of 25 and the procedureconcludes, featuring the final customised value function as seen on Figure A.17.


Figure A.17: Customised Value Function for “Maintenance Cost” in ElectioVis


Table A.6: Standardised Scores for Maintenance Costs


Car 1 200 0

Car 2 100 50− (100− 95) ∗ (50− 25)

147.5− 95= 47.61

Car 3 150 25− (150− 147.5) ∗ (25− 0)

200− 147.5= 23.81

Car 4 50 100

A.1.5.6 Colour

For “Colour” the default proportional linear shape assigns a value of 0 to the worstperformance (“Ugly”) and a value of 100 to the best performance (“Nice”). In turn,“Normal” receives a score of 50.



Figure A.18: Value Function for “Colour” in ElectioVis

Let us assume that the user wants to keep the function as it is. Standardised scoresmay now be calculated as shown in Table A.7.

Table A.7: Standardised Scores for Colour


Car 1 Ugly 0

Car 2 Nice 100

Car 3 Normal 50

Car 4 Nice 100

With this, we conclude the Value Function elicitation phase. Table A.8 summarisedthe standardised scores for all of the objectives.

Table A.8: Performance AssessmentsSpeed Price Comfort Security Maintenance Colour

Car 1 100 0 100 50 0 0Car 2 53.73 37.5 60 47.61 100 100Car 3 0 0 80 25 23.81 50Car 4 37.31 100 0 0 100 100

A.1.6 Trade-offs

The purpose of this final step is to elicit the relative importance of each objectiveunder the form of weights. This is to allow the calculation of a weighted average score for


each alternative.

Figure A.19 shows the initial Trade-offs screen in ElectioVis.

Figure A.19: Initial Trade-offs screen in ElectioVis

At this point, the user is able to simultaneously see all the graphical representationsof the differences between the best and the worst alternative as far as each objective isconcerned. In particular, the best and worst alternatives appear highlighted in green andred, respectively.

After having a look at all the charts, the user must decide what is the most significantdifference and is asked to rate it with a score (a score of 100 is advised for the differencedeemed as the most important).

After assigning the numeric score, the checkbox of the relevant objective may be ac-tivated to hide the chart and focus on the remaining ones. Let us assume that, from theuser’s perspective, the most significant difference is that of “Price” and assigns a score of100 to it.

Figure A.20 shows the Trade-offs screen after hiding “Price”, that was already assigneda score of 100.


Figure A.20: Trade-offs screen in ElectioVis after assessing “Price”

Secondly, let us imagine that the user decides that “Security” registers the second mostimportant difference and assigns a score of 90 to it. This is depicted in Figure A.21.

Figure A.21: Trade-offs screen in ElectioVis after assessing “Security”

Thirdly, let us imagine that the user decides that “Comfort” witnesses the third mostimportant difference and assigns a score of 80 to it. This is depicted in Figure A.22.


Figure A.22: Trade-offs screen in ElectioVis after assessing “Comfort”

Now, if the user decides that “Colour” witnesses the fourth most important differenceand assigns a score of 70 to it, the screen will take the shape of Figure A.23.

Figure A.23: Trade-offs screen in ElectioVis after assessing “Colour”

Moving on, if the difference in “Maintenance Costs” is considered to be as important asthe difference in “Colour”, a score of 70 may be assigned to it. This is depicted in FigureA.24.


Figure A.24: Trade-offs screen in ElectioVis after assessing “Maintenance Costs”

Finally, the difference regarding “Speed” is deemed as the least important and a scoreof 50 is assigned to it. This is depicted in Figure A.25.

Figure A.25: Trade-offs screen in ElectioVis after assessing “Speed”

Table A.9 summarises the absolute score for each objective and the resulting normalisedscores.


Table A.9: Absolute and Normalised Scores by ObjectiveAbsolute score Normalised Score

Price 100 100 / 460 = 21.74 %Security 90 90 / 460 = 19.57 %Comfort 80 80 / 460 = 17.39 %Colour 70 70 / 460 = 15.22 %

Maintenance 70 70 / 460 = 15.22 %Speed 50 50 / 460 = 10.87 %TOTAL 460 -

A.1.7 Results

If we multiply the scores depicted in Table A.8 by the weights depicted in Table A.9,we obtain Table A.10.

Table A.10: Weighted Performance AssessmentsSpeed Price Comfort Security Maintenance Colour TOTAL

Car 1 10.8696 0 17.3913 9.7826 0 0 38.0435Car 2 5.8403 8.1522 10.4348 19.5652 7.2464 15.2174 66.4563Car 3 0 0 13.9130 4.8913 3.6232 7.6087 30.0362Car 4 4.0558 21.7391 0 0 15.2174 15.2174 56.2297

Figure A.26 depicts the Results by Objective in ElectioVis.

Figure A.26: Results by Objective in ElectioVis

The column “Total” of Table A.10 is the unified score of each alternative. This showsthat “Car 2” is the best, “Car 4” is the second best, “Car 1” is in the third position and“Car 3” is in the fourth position.

Figure A.27 depicts the Results by Alternative in ElectioVis.

A.2. ELECTRE III 87

Figure A.27: Results by Alternative in ElectioVis

A.2 ELECTRE III


Let us imagine that a decision maker wants a pet but is not sure which type he wouldlike to adopt. His decision-making scenario has the following features:

• He would like to obtain an ordinal ranking of the potential pets he is analysing.

• If an animal is very good in one aspect but very bad in another aspect, he does notwant those contradictory performances to be compensated.

• He would like to compare potential pets using numbers for some objectives andcategories for some other objectives.

• He is aware that, if more pets are later added after completing one process of analysis,rankings might eventually change (in other words, he does not expect “indepen-dence to third alternatives”).

Having to choose between Multi-Attribute Value Theory, Analytic Hierarchy Processand ELECTRE III, the latter, i.e. ELECTRE III, seems to be the most suitable methodfor this decision-making scenario.

In the Method Choice screen, we can see that ElectioVis recognises ELECTRE III asthe only method with no vetoes and the highest number of approvals, as seen on FigureA.28.



A.2.2 Criteria


• Maximising the Quality of the interaction with pet (measured qualitatively bymeans of one of the following categories: “Neutral”, “Good” or “Very good”).

• Maximising its Visual Appeal (measured qualitatively by means of one of thefollowing categories: “Neutral”, “Good” or “Very good”).

• Minimising the expectedMaintenance Cost of it (measured quantitatively in Europer month).

• Maximising the expected Life expectancy of it (measured quantitatively in years).

• Minimising its periodic Dependency (measured quantitatively in hours per weekof dedication).

Figure A.29 shows how the information about criteria and thresholds (which are ex-plained in the next subsection) is entered in ElectioVis.

Figure A.29: Criteria in ElectioVis

A.2. ELECTRE III 89

A.2.3 Thresholds

A.2.3.1 Indifference Thresholds

They represent the maximum differences that may occur (regarding each criterion) forthe decision maker to feel indifferent between two alternatives. Let us assume that:

• The decision maker is sensitive about any difference between alternatives in Inter-action Quality or Visual appeal and will prefer the alternatives that show theslightest advantage, no matter how small it is.

• Two pets will be deemed as indistinguishable as long as they have a difference of upto 10 EUR in their Maintenance Cost.

• Similarly, two pets will be deemed as indistinguishable as long as they have a differ-ence of up to 2 years in their Life Expectancy.

• Finally, as far as Dependency is concerned, two pets will be deemed as indistin-guishable as long as they have a difference of up to 2 hours in the amount of timethey weekly require.

A.2.3.2 Preference Thresholds

They represent the minimum differences that should exist between two alternatives forthe decision maker to strongly prefer one of them (as opposed to a slight preference). Letus assume that:

• As far as Interaction Quality and Visual appeal are concerned, the decisionmaker will strongly prefer an alternative with the smallest difference (no slight orweak preference exists for these two objectives).

• A difference of at least 20 EUR in Maintenance Cost will lead the decision makerto strongly prefer the cheaper alternative.

• A difference of at least 5 years in Life Expectancy will mean a strong preferencefor the most durable pet.

• Finally, as far as Dependency is concerned, a difference of at least 4 hours willalso mark a strong preference.

A.2.3.3 Veto Thresholds

It might happen that one alternative is better than another one in all criteria but one.However, if the difference in that criterion is too important, the decision maker might wantto invalidate the other contradictory assessments. The veto threshold sets the minimumdifference that should exist for doing so. Let us assume that:

• As far as Interaction Quality and Visual appeal are concerned, a difference oftwo categories (i.e. from “Neutral” to “Very good”) is significant enough to overrulea majority of performances in the opposite direction.


• A difference of at least 50 EUR in Maintenance Cost can also overrule the ma-jority.

• A difference of at least 10 years in Life Expectancy will also produce the sameoutcome.

• Finally, as far as Dependency is concerned, a difference of at least 10 hours willalso trigger a veto.

Table A.11 summarises Indifference, Preference and Veto thresholds for each criterion.

Table A.11: Thresholds for the CriteriaIndifference (q) Preference (p) Veto (v)

Interaction 0 1 2Visual Appeal 0 1 2Maintenance Cost 10 20 50Life Expectancy 2 5 10Dependency 2 4 10

A.2.4 Alternatives

We will assume that the decision maker wants to choose a pet type among a group offour alternatives:

• Cat

• Dog

• Fish

• Bird

A.2.5 Performance Assessments

The next step involves gathering the values of the attributes defined in the Criteriascreen for each of the alternatives.

Table A.12 summarises the values that will be used for this example:

Table A.12: Performance AssessmentsInteraction Visual Appeal Maintenance Life Expectancy Dependency

Bird Neutral Good 17 20 3Cat Good Good 55 14 6Dog Very good Very good 58 12 10Fish Neutral Neutral 3 20 1

Figure A.30 shows how performance assessments were input into ElectioVis.

A.2. ELECTRE III 91

Figure A.30: Performance Assessments in ElectioVis

A.2.6 Trade-offs

We will use Simo’s White Cards procedure, revised in [Figueira 2002], in order to elicitthe trade-offs among objectives.

Let us assume that the decision maker sorts objectives according to the followingrankings:

Table A.13: Ranking of ObjectivesRank Criteria

1 “Interaction Quality”[White card][White card]

2 “Visual Appeal” and “Life Expectancy”[White card]

3 “Dependancy”4 “Maintenance Cost”

In addition, let us consider that, for the decision maker, Interaction Quality is 5times more important than Maintenance Cost (this is the parameters z of the modelwhich will be later referred to).

The relevant calculations are shown in Table A.14.

Table A.14: Ranking of Objectives CalculationsRank (r) Criteria e’r er

∑r−11 er k(r) Total Weight

1 { Maintenance Cost } 0 1 0 1 1 7.32 %2 { Dependancy } 1 2 1 1.667 1.667 12.20 %3 { Visual Appeal, Life Expectancy } 2 3 3 3 6 21.95 %4 { Interaction Quality } 0 - 6 5 5 36.59 %

TOTAL 6 13.667

Where:

• e’r is the number of White cards between ranks r and r+1.

• er = e’r + 1 ∀ r = 1, ..., n - 1.


• e0 = 0

• e =∑n−1

r=1 er = 6

• u =z − 1

e=

5− 1

6= 0.667

• k(r) = 1 + u (e0 + ... + er−1)

• Total = k(r) * |Criteria|

• Weight =k(r)∑n1 Total

* 100

Figure A.31 shows the White cards screen in ElectioVis where the ranking of objec-tives is input and the weights are interactively calculated.

Figure A.31: Trade-offs in ElectioVis

A.2.7 Calculation of Concordance

As described in [Figueira 2005a, Chapter 4], the Concordance Index is calculated asfollows:

C(a, b) =1∑n

j=1wj

∑nj=1wjcj(a, b)

For criteria to be maximised, cj(a, b) is calculated as follows:

• Case 1: gj(a) + qj(gj(a)) ≥ gj(b) ∴ cj(a, b) = 1

• Case 2: gj(a) + pj(gj(a)) ≤ gj(b) ∴ cj(a, b) = 0

• Case 3: Otherwise:gj(a)− gj(b) + pj(gj(a))

pj(gj(a))− qj(gj(a))

For criteria to be minimised, cj(a, b) is calculated as follows:

A.2. ELECTRE III 93

• Case 1: gj(a) - qj(gj(a)) ≤ gj(b) ∴ cj(a, b) = 1

• Case 2: gj(a) - pj(gj(a)) ≥ gj(b) ∴ cj(a, b) = 0

• Case 3: Otherwise:gj(b)− gj(a) + pj(gj(a))

pj(gj(a))− qj(gj(a))

We will now move on to the details of the calculations.

A.2.7.1 C(Bird, Cat)

• Interaction (maximisation): cj(a, b) = 0 (Case 2)

• Visual Appeal (maximisation): cj(a, b) = 1 (Case 1)

• Maintenance Cost (minimisation): cj(a, b) = 1 (Case 1)

• Life Expectancy (maximisation): cj(a, b) = 1 (Case 1)

• Dependency (minimisation): cj(a, b) =3− 6 + 4

4− 2= 1 (Case 3)

C(Bird, Cat) = 0.3659 * 0 + 0.2195 * 1 + 0.0732 * 1 + 0.2195 * 1 + 0.1220 * 1

C(Bird, Cat) = 0.6342

A.2.7.2 C(Bird, Dog)





• Dependency (minimisation): cj(a, b) = 1 (Case 1)

C(Bird, Dog) = 0.3659 * 0 + 0.2195 * 0 + 0.0732 * 1 + 0.2195 * 1 + 0.1220 * 1

C(Bird, Dog) = 0.4147

A.2.7.3 C(Bird, Fish)



• Maintenance Cost (minimisation): cj(a, b) =3− 17 + 20

20− 10= 0.6 (Case 3)




C(Bird, Fish) = 0.3659 * 1 + 0.2195 * 1 + 0.0732 * 0.6 + 0.2195 * 1 + 0.1220 * 1

C(Bird, Fish) = 0.9708

A.2.7.4 C(Cat, Bird)





• Dependency (minimisation): cj(a, b) =3− 6 + 4

4− 2= 0.5 (Case 3)

C(Cat, Bird) = 0.3659 * 1 + 0.2195 * 1 + 0.0732 * 0 + 0.2195 * 0 + 0.1220 * 0.5

C(Cat, Bird) = 0.6464

A.2.7.5 C(Cat, Dog)






C(Cat, Dog) = 0.3659 * 0 + 0.2195 * 0 + 0.0732 * 1 + 0.2195 * 1 + 0.1220 * 1

C(Cat, Dog) = 0.4147

A.2.7.6 C(Cat, Fish)






C(Cat, Fish) = 0.3659 * 1 + 0.2195 * 1 + 0.0732 * 0 + 0.2195 * 0 + 0.1220 * 0

C(Cat, Fish) = 0.5854

A.2. ELECTRE III 95

A.2.7.7 C(Dog, Bird)






C(Dog, Bird) = 0.3659 * 1 + 0.2195 * 1 + 0.0732 * 0 + 0.2195 * 0 + 0.1220 * 0

C(Dog, Bird) = 0.5854

A.2.7.8 C(Dog, Cat)






C(Dog, Cat) = 0.3659 * 1 + 0.2195 * 1 + 0.0732 * 1 + 0.2195 * 1 + 0.1220 * 0

C(Dog, Cat) = 0.878

A.2.7.9 C(Dog, Fish)






C(Dog, Fish) = 0.3659 * 1 + 0.2195 * 1 + 0.0732 * 0 + 0.2195 * 0 + 0.1220 * 0

C(Dog, Fish) = 0.5854


A.2.7.10 C(Fish, Bird)






C(Fish, Bird) = 0.3659 * 1 + 0.2195 * 0 + 0.0732 * 1 + 0.2195 * 1 + 0.1220 * 1

C(Fish, Bird) = 0.7805

A.2.7.11 C(Fish, Cat)






C(Fish, Cat) = 0.3659 * 0 + 0.2195 * 0 + 0.0732 * 1 + 0.2195 * 1 + 0.1220 * 1

C(Fish, Cat) = 0.4147

A.2.7.12 C(Fish, Dog)






C(Fish, Dog) = 0.3659 * 0 + 0.2195 * 0 + 0.0732 * 1 + 0.2195 * 1 + 0.1220 * 1

C(Fish, Dog) = 0.4147

Table A.15 depicts the values of the Concordance Matrix.

A.2. ELECTRE III 97

Table A.15: Concordance MatrixBird Cat Dog Fish

Bird 1 0.6342 0.4147 0.9708Cat 0.6464 1 0.4147 0.5854Dog 0.5854 0.878 1 0.5854Fish 0.7805 0.4147 0.4147 1

A.2.8 Calculation of Discordance

For criteria to be maximised, Dj(a, b) is calculated as follows:

• Case 1: gj(b) ≤ gj(a) + pj(gj(a)) ∴ Dj(a, b) = 0

• Case 2: gj(b) ≥ gj(a) + vj(gj(a)) ∴ Dj(a, b) = 1

• Case 3: Otherwise:gj(b)− gj(a)− pj(gj(a))

vj(gj(a))− pj(gj(a))

For criteria to be minimised, Dj(a, b) is calculated as follows:

• Case 1: gj(b) ≥ gj(a) - pj(gj(a)) ∴ Dj(a, b) = 0

• Case 2: gj(b) ≤ gj(a) - vj(gj(a)) ∴ Dj(a, b) = 1

• Case 3: Otherwise:gj(a)− gj(b)− pj(gj(a))

vj(gj(a))− pj(gj(a))

We will now move on to the details of the calculations, assuming that:

• gj(Neutral) = 1

• gj(Good) = 2

• gj(Very good) = 3

A.2.8.1 C(Bird, Cat)

• Interaction (maximisation): Dj(a, b) = 0 (Case 1)

• Visual Appeal (maximisation): Dj(a, b) = 0 (Case 1)

• Maintenance Cost (minimisation): Dj(a, b) = 0 (Case 1)

• Life Expectancy (maximisation): Dj(a, b) = 0 (Case 1)

• Dependency (minimisation): Dj(a, b) = 0 (Case 1)

A.2.8.2 C(Bird, Dog)


As one of the Discordance indexes is 1, there is no need to calculate the others.


A.2.8.3 C(Bird, Fish)






A.2.8.4 C(Cat, Bird)



• Maintenance Cost (minimisation): Dj(a, b) =55− 17− 20

50− 20= 0.6 (Case 3)

• Life Expectancy (maximisation): Dj(a, b) =20− 14− 5

10− 5= 0.2 (Case 3)


A.2.8.5 C(Cat, Dog)






A.2.8.6 C(Cat, Fish)





A.2. ELECTRE III 99

A.2.8.7 C(Dog, Bird)



• Maintenance Cost (minimisation): Dj(a, b) =58− 17− 20

50− 20= 0.7 (Case 3)

• Life Expectancy (maximisation): Dj(a, b) =20− 12− 5

10− 5= 0.6 (Case 3)

• Dependency (minimisation): Dj(a, b) =10− 3− 4

10− 4= 0.5 (Case 3)

A.2.8.8 C(Dog, Cat)






A.2.8.9 C(Dog, Fish)





A.2.8.10 C(Fish, Bird)







A.2.8.11 C(Fish, Cat)






A.2.8.12 C(Fish, Dog)



A.2.9 Calculation of Credibility Score

The Credibility Score is calculated as follows:

• Case 1: Dj(a, b) ≤ C(a, b),∀j ∴ S(a, b) = C(a, b)

• Case 2: S(a,b) = C(a,b)∏j∈Ψ(a,b)

1−Dj(a, b)

1− C(a, b)where Ψ(a, b) is the set of criteria for which Dj(a, b) > C(a, b)

Now, moving on to the calculations:

• S(Bird, Cat) = C(Bird, Cat) = 0.6342Case 1, as the greatest discordance index is 0.

• S(Bird, Dog) = 0Case 2, having a discordance index of 1 for “Interaction”.

• S(Bird, Fish) = C(Bird, Fish) = 0.9708Case 1, as the greatest discordance index is 0.

• S(Cat, Bird) = C(Cat, Bird) = 0.6464Case 1, as the greatest discordance index is 0.6.

• S(Cat, Dog) = C(Cat, Dog) = 0.4147Case 1, as the greatest discordance index is 0.

• S(Cat, Fish) = 0Case 2, having a discordance index of 1 for “Maintenance Cost”.

• S(Dog, Bird) = 0.5854 *1− 0.7

1− 0.5854*

1− 0.6

1− 0.5854= 0.4087

Case 2, where the discordance indexes for “Maintenance Cost” and “Life Expectancy”exceed the concordance index.

A.2. ELECTRE III 101

• S(Dog, Cat) = C(Dog, Cat) = 0.878Case 1, as the greatest discordance index is 0.

• S(Dog, Fish) = 0Case 2, having a discordance index of 1 for “Maintenance Cost”.

• S(Fish, Bird) = C(Fish, Bird) = 0.7805Case 1, as the greatest discordance index is 0.

• S(Fish, Cat) = C(Fish, Cat) = 0.4147Case 1, as the greatest discordance index is 0.

• S(Fish, Dog) = 0Case 2, having a discordance index of 1 for “Interaction”.

Table A.16 summarises all credibility index values.

Table A.16: Credibility MatrixBird Cat Dog Fish

Bird 1 0.6342 0 0.9708Cat 0.6464 1 0.4147 0Dog 0.4087 0.878 1 0Fish 0.7805 0.4147 0 1

A.2.10 Ascending Distillation

A.2.10.1 Distillation 1

Stage 1

λ0 = maxS(a, b) = 0.9708 (the greatest value from the Credibility Matrix)

S(λ0) = 0.3 - 0.15 λ0 = 0.3 - 0.15 * 0.9708 ≈ 0.1544

λ0 − S(λ0) ≈ 0.9708 - 0.1544 ≈ 0.8164

λ1 = maxS(a,b)<0.8164 S(a,b) = 0.7805

Alternative “a” outranks alternative “b” if and only if:

• S(a,b) > λ1, and

• S(a,b) - S(b,a) > S(λ)

For each alternative, we will count how many times it outranks the other alterna-tives (strengths) and how many times it is outranked by the other alternatives (weak-nesses). Then, we can get the qualification of each alternative, by subtracting weaknessesto strengths. Table A.17 shows qualifications for this step.


Table A.17: Qualifications for each alternative - 1.1Strengths Weaknesses Qualification

Bird 1 0 1Cat 0 1 -1Dog 1 0 1Fish 0 1 -1

The minimum qualification is -1, so we move on to Stage 2 with the alternatives “Cat”and “Fish”.

Stage 2

S(λ1) = 0.3 - 0.15 λ1 ≈ 0.3 - 0.15 * 0.7805 ≈ 0.1829

λ1 − S(λ1) = 0.7805 - 0.1829 ≈ 0.5976

λ2 = maxS(a,b)<0.5976 S(a,b) = 0.4146

We proceed as described before in order to determine outranking relationships betweenalternatives. This allows us to calculate strengths, weaknesses and qualifications. TableA.18 shows that we need to advance to the next stage because there is still a tie between“Cat” and “Fish”.


Cat 0 0 0Fish 0 0 0

Stage 3

S(λ2) = 0.3 - 0.15 λ2 = 0.3 - 0.15 * 0.4146 ≈ 0.2378

λ2 − S(λ2) ≈ 0.4146 - 0.2378 ≈ 0.1768

λ3 = maxS(a,b)<0.1768 S(a,b) = 0

Table A.19 shows the new qualification values.


Cat 0 1 -1Fish 1 0 1


The minimum qualification is -1 and it is linked to “Cat”. Therefore, so far, in the as-cending distillation we have “Cat” and then “Fish”. We now move on to the next distillationto rank the remaining alternatives.


Stage 1

λ0 = maxS(a, b) = 0.4087 (the greatest value of the Credibility Matrix, excluding“Cat” and “Fish”, of course).

S(λ0) = 0.3 - 0.15 λ0 = 0.3 - 0.15 * 0.4087 ≈ 0.2387

λ0 − S(λ0) ≈ 0.4087 - 0.2387 ≈ 0.17

λ1 = maxS(a,b)<0.17 S(a,b) = 0

Table A.20 shows the qualifications for each alternative.


Dog 1 0 1Bird 0 1 -1

The minimum qualification is -1 and it is linked to “Bird”. Therefore, the ascendingdistillation finishes with the following inverted ranking (from the least preferred to themost preferred):

1. Cat

2. Fish

3. Bird

4. Dog

A.2.11 Descending Distillation


Stage 1

λ0 = maxS(a, b) = 0.9708 (the greatest value from the Credibility Matrix)

S(λ0) = 0.3 - 0.15 λ0 = 0.3 - 0.15 * 0.9708 ≈ 0.1544

λ0 − S(λ0) ≈ 0.9708 - 0.1544 ≈ 0.8164


λ1 = maxS(a,b)<0.8164 S(a,b) = 0.7805



Bird 1 0 1Cat 0 1 -1Dog 1 0 1Fish 0 1 -1

The maximum qualification is 1, so we move on to Stage 2 with the alternatives “Bird”and “Dog”.

Stage 2

S(λ1) = 0.3 - 0.15 λ1 = 0.3 - 0.15 * 0.7805 ≈ 0.1829

λ1 − S(λ1) ≈ 0.7805 - 0.1829 ≈ 0.5976

λ2 = maxS(a,b)<0.5976 S(a,b) = 0.4087

Table A.22 shows that we need to advance to the next stage because there is still a tie.


Bird 0 0 0Dog 0 0 0

Stage 3

S(λ2) = 0.3 - 0.15 λ2 ≈ 0.3 - 0.15 * 0.4087 ≈ 0.1829

λ2 − S(λ2) ≈ 0.4087 - 0.1829 ≈ 0.2387

λ3 = maxS(a,b)<0.2387 S(a,b) = 0

Table A.23 shows the qualification for each alternative.



Bird 0 1 -1Dog 1 0 1

The maximum qualification is 1 and it is linked to “Dog”. Therefore, so far, in thedescending distillation we have “Dog” and then “Bird”. We now move on to the nextdistillation.


Stage 1

λ0 = maxS(a, b) = 0.4147 (the greatest value of the Credibility Matrix excluding“Dog” and “Bird”, of course)

S(λ0) = 0.3 - 0.15 λ0 ≈ 0.3 - 0.15 * 0.4147 ≈ 0.2378

λ0 − S(λ0) ≈ 0.4147 - 0.2378 ≈ 0.1769

λ1 = maxS(a,b)<0.1769 S(a,b) = 0



Fish 1 0 1Cat 0 1 -1

The maximum qualification is 1 and it is linked to “Fish”. Therefore, the descend-ing distillation finishes with the following ranking (from the most preferred to the leastpreferred):

1. Dog

2. Bird

3. Fish

4. Cat

A.2.12 Results

The complete ranking is obtained by calculating the intersection between the ascendingand the descending distillations. In particular, an alternative is preferred to anotheralternative if and only if:


1. It is preferred either in the ranking of the ascending distillation or in the ranking ofthe descending distillation.

2. The remaining ranking does not contradict such a preference (in words, it is preferredor indifferent in the other ranking).

In the example we are analysing, the ranking of the ascending distillation and theranking of the descending distillation are totally compatible. Therefore, the final rankingturns out to be:

1. Dog

2. Bird

3. Fish

4. Cat

Figure A.32 shows how the ranking is depicted in ElectioVis Results screen.

Figure A.32: Results in ElectioVis

A.3 Analytic Hierarchy Process


Let us imagine that a person wants to make a decision on her career. This hypo-thetical decision-making scenario has the following features:

• She would like to obtain a numerical score for each of the jobs she is analysing.

• If a job is very good in one aspect but very bad in another aspect, shemight acceptthat those contradictory performances are compensated, but that does notrepresent a mandatory requirement.

• She would like to do comparisons such as “Job 1 is strongly better in Salary thanJob 2”, “Job 3 required the same dedication than Job 2”, etc.

A.3. Analytic Hierarchy Process 107

• She is not worried about the consistency of assessments in case more poten-tial jobs are later added after completing one process of analysis (in other words,“independence to third alternatives” is not mandatory).

Having to choose between Multi-Attribute Value Theory, Analytic Hierarchy Processand ELECTRE III, the second of them, i.e. Analytic Hierarchy Process, seems to bethe most suitable method for this decision-making scenario.

In the Method Choice screen, we can see that ElectioVis recognises Analytic HierarchyProcess as the only method with no vetoes and the highest number of approvals, as seenon Figure A.33.


A.3.2 Objectives


• Maximising the Salary of the job, which represents a Benefit.

• Maximising the Stimulation provided by the tasks to be performed on the job,which represents a Benefit.

• Maximising the opportunities of future Professional Development, which repre-sents a Benefit.

• Minimising the amount of Dedication Required, so as to improve her work/lifebalance. The amount of dedication required therefore represents a Cost.

Figure A.34 shows how these criteria are entered in ElectioVis.


Figure A.34: Criteria in ElectioVis

A.3.3 Alternatives

We will assume that the decision maker wants to decide whether she will keep hercurrent job or if she will accept one of the two job offers she has received. Therefore, thelist of alternatives is:

• Current Job

• Opportunity 1

• Opportunity 2

Figure A.35 shows how alternatives were input into ElectioVis.

Figure A.35: Alternatives in ElectioVis

A.3.4 Objectives Comparison

The user now needs to express her priorities as far as the objectives are concerned andthis is not only done in an ordinal way but preferences need to be quantified. In order todo so, the following scale is used:

• 9: “A” is extremely more important than “B”.

• 7: “A” is very strongly more important than “B”.

• 5: “A” is strongly more important than “B”.

• 3: “A” is moderately more important than “B”.


• 1: “A” and “B” are equally important.

• 13 : “B” is moderately more important than “A”.

• 15 : “B” is strongly more important than “A”.

• 17 : “B” is very strongly more important than “A”.

• 19 : “B” is extremely more important than “A”.

Intermediate values may also be used if necessary:

• 8: “A” is very strongly to extremely more important than “B”.

• 6: “A” is strongly to very strongly more important than “B”.

• 4: “A” is moderately to strongly more important than “B”.

• 2: “A” is equally to moderately more important than “B”.

• 12 : “B” is equally to moderately more important than “A”.

• 14 : “B” is moderately to strongly more important than “A”.

• 16 : “B” is strongly to very strongly more important than “A”.

• 18 : “B” is very strongly to extremely more important than “A”.

Let us assume that the decision maker expressed the following comparisons:

• “Professional Development” is extremely more important than “Dedication Re-quired ”.

• “Dedication Required ” and “Salary” are equally important.

• “Stimulating tasks” is strongly more important than “Dedication Required ”.

• “Professional Development” is very strongly more important than “Salary”.

• “Professional Development” is strongly more important than “Stimulating tasks”.

• “Stimulating tasks” is strongly more important than “Salary”.

Figures A.36 and A.37 show the objective comparisons of this example in ElectioVis.


Figure A.36: Objective Comparisons in ElectioVis (verbally)

Figure A.37: Objective Comparisons in ElectioVis (as a matrix)

In order to obtain the objectives weights, we need to calculate the Eigen Vector. Thematrix depicted in Table A.25 represents our starting point.

Table A.25: Objectives Comparison Matrix

1 19 1 1

5

9 1 7 5

1 17 1 1

5

5 15 5 1

We first square the matrix and obtain the result shown in A.26.


Table A.26: Objectives Comparison Matrix Squared

4 0.40507936507936504 3.7777777777777777 1.1555555555555554

50 4 48 13.2

4.285714285714286 0.4368253968253968 4 1.314285714285714

16.8 1.6698412698412697 16.4 4

We now sum each row and obtain the results depicted in Table A.27.

Table A.27: Sum of rows

9.338412698412696

115.2

10.036825396825396

38.869841269841274

Moving on, we normalise values in A.27. In order to do so, we sum all of them toget the total and then divide each individual value by the total. The Provisional Weightsarising from this first iteration are shown in A.28.

Table A.28: Provisional Weights (Iteration 1)

5.3840747357015245 %

66.41871906756084 %

5.786745541335987 %

22.410460655401646 %

Now, we proceed with a second iteration. In order to do so, we square the matrixdepicted in Table A.26, we sum the rows of the resulting matrix and then we normalise.The Provisional Weights arising from this second iteration are shown in A.29.



5.674591960135005 %

65.38611700470891 %

6.1754018351991166 %

22.763889199956983 %

The sum of differences between the first and the second iteration is slightly higherthan 2.06 %. Therefore, a new iteration is performed. ElectioVis will perform up to 10iterations but, if the difference between two iterations is less than 0.0000000000001 %, itwill stop sooner. The Provisional Weights arising from this third iteration are shown inA.30.


5.6623822249127466 %

65.43856441715561 %

6.1599547731034154 %

22.73909858482822 %

Now, the sum of differences between the results of the second and the third iterationis 0.10489482489342006 %. The results of the fourth iteration are depicted in Table A.31.


5.6623626211084695 %

65.43868074520205 %

6.1599331204184024 %

22.73902351327108 %

The sum of differences is very small now (0.0002326560928703203 %) but it is stillgreater than 0.0000000000001 %. The results of the fifth iteration are depicted in TableA.32.



5.662362621087045 %

65.43868074552873 %

6.159933120414303 %

22.739023512969928 %

The sum of differences keeps on decreasing (0.0000000006533606988767815 %) but itis still greater than 0.0000000000001 %. The results of the sixth iteration are depicted inTable A.33.


5.662362621087044 %

65.43868074552872 %

6.159933120414303 %

22.739023512969922 %

The sum of differences is now 0.00000000000001734723475976807 % so the results ofiteration 6 may be deemed as the final weights of the objectives “Dedication Required ”,“Professional Development”, “Salary” and “Stimulating tasks” respectively.

We will now check if the weights are consistent. In order to do so, we multiply theoriginal matrix (Table A.25) by the weights vector (Table A.33), thus obtaining the vectorshown in Table A.34.

Table A.34: Consistency Check - Step 1

0.236410649713763

2.7321459374306185

0.2571848340774229

0.9493823836958241

Then, we multiply the vector in Table A.34 by the inverse of the weights vector (TableA.33), obtaining a new vector, which is shown in Table A.35.


Table A.35: Consistency Check - Step 2

4.175123804917629

4.175123804917629

4.175123804917629

4.175123804917629

The average of the vector shown in Table A.35 is 4.175123804917629 (λ). Theconsistency index is calculated through the following formula:

λ− |weights||weights| − 1

=4.175123804917629− 4

4− 1= 0.05837460163920966666666666666667

Finally, the consistency ratio is obtained by dividing the consistency index by theexpected consistency of a random matrix. Instead of using Saaty’s random indexes, wewill use the revised values presented by [Donegan 1991]. For a matrix of order 4, Donegansuggests an index of 0.8045. Therefore:

0.05837460163920966666666666666667

0.8045= 0.0725601014781971

As 7.26 % is less than the maximum advised value of 10 %, we can assume thatconsistency is within acceptable values.

Figure A.38 shows objectives weights and the consistency ratio in ElectioVis.

Figure A.38: Objective Weights


A.3.5 Alternatives Comparison

A.3.5.1 According to Dedication Required


• “Opportunity 1 ” is strongly better than “Current Job”.

• “Current Job” is moderately better than “Opportunity 2 ”.

• “Opportunity 1 ” is extremely better than “Opportunity 2 ”.

This is translated into the matrix that can be seen in Figure A.39.

Figure A.39: Matrix of Comparison of Alternatives according to Dedication Required

After performing the calculations as we have done for the objectives weights, thesystem obtains the scores by alternative for the objective “Dedication Required ”, whichare depicted in Figure A.40.

Figure A.40: Score by Alternative for Dedication Required


A.3.5.2 According to Professional Development


• “Opportunity 1 ” is moderately to strongly better than “Current Job”.

• “Current Job” is strongly better than “Opportunity 2 ”.



Figure A.41: Matrix of Comparison of Alternatives according to Professional Development

After performing the calculations as we have done for the objectives weights, the systemobtains the scores by alternative for the objective “Professional Development”, which aredepicted in Figure A.42.

Figure A.42: Score by Alternative for Professional Development


A.3.5.3 According to Salary


• “Current Job” is strongly better than “Opportunity 1 ”.

• “Current Job” is very strongly to extremely better than “Opportunity 2 ”.

• “Opportunity 1 ” is moderately better than “Opportunity 2 ”.


Figure A.43: Matrix of Comparison of Alternatives according to Salary

After performing the calculations as we have done for the objectives weights, the systemobtains the scores by alternative for the objective “Salary”, which are depicted in FigureA.44.

Figure A.44: Score by Alternative for Salary


A.3.5.4 According to Stimulating Tasks


• “Opportunity 1 ” is moderately better than “Current Job”.

• “Current Job” is strongly to very strongly better than “Opportunity 2 ”.



Figure A.45: Matrix of Comparison of Alternatives according to Stimulating Tasks

After performing the calculations as we have done for the objectives weights, the systemobtains the scores by alternative for the objective “Stimulating Tasks”, which are depictedin Figure A.46.

Figure A.46: Score by Alternative for Stimulating Tasks

Table A.36 summarises the scores by alternative according to each criterion.


Table A.36: Summary of Scores per AlternativeCurrent Job Opportunity 1 Opportunity 2

Dedication Required 17.82 % 75.14 % 7.04 %Professional Development 23.11 % 70.85 % 6.03 %Salary 74.11 % 18.30 % 7.52 %Stimulating Tasks 27.85 % 66.31 % 5.85 %

A.3.6 Results

If we multiply the scores per alternative depicted in Table A.36 by the objectivesweights depicted in Table A.33, we get the weighted average score of each alternative,which is shown in Table A.37.

Table A.37: Weighed Average Score per AlternativeCurrent Job Opportunity 1 Opportunity 2

Dedication Required 1.0090 % 4.2547 % 0.3986 %Professional Development 15.1229 % 46.3633 % 3.9460 %Salary 4.5651 % 1.1273 % 0.4632 %Stimulating Tasks 6.3328 % 15.0782 % 1.3302 %TOTAL 27.0298 % 66.8235 % 6.138 %

This allows us to conclude that “Opportunity 1 ” seems to be the most valued accordingto the decision maker’s preferences, distantly followed by the alternative “Current Job”,which represents the Statu quo.

Figure A.47 shows a graphical comparison of the alternatives which is prepared on thebasis of the scores discussed.

Figure A.47: Results by Alternative

Furthermore, Figure A.48 shows a graphical comparison by objective.


Figure A.48: Results by Objective

And, finally, Figure A.49 shows the Cost-Benefit ratio of each alternative. “Profes-sional Development”, “Salary” and “Stimulating Tasks” are regarded as Benefit criteria(as defined in Figure A.34), whereas “Dedication Required ” is defined as a Cost criterion.

Figure A.49: Cost-Benefit Analysis

This shows that “Opportunity 1 ” not only provides the highest benefits (in terms of“Professional Development”, “Salary” and “Stimulating Tasks”) but it also has the lowestcost (in terms of “Dedication Required ”).

Appendix B

Bisection Gap Calculation Algorithm Tests

This Appendix evidences the tests performed to validate the Bisection Gap Calculation Algorithm defined in 4.1.2.2.

We can see that gaps are always consistent with the scale of the reference points, no matter if they are expressed in thousands,hundreds or simply decimal fractions and regardless of positive or negative signs. They also keep close to rounded numbers asintended, at the same time they represent between one fifth and one tenth of the difference between the “Lower Bound ” and the“Assessment Point” (as the number of resulting chunks ranges from 5 to 10). Furthermore, in all cases, gaps get smoother as the“Assessment Point” gets closer to the “Lower Bound ” over the course of comparisons.

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10

cand1 cand2 cand3 cand4

Roun-dedGap

Correc-tedGap

FinalGap

Chunks

1 100 150 125 2.5 100 50 10 5 5 5 5 5100 150 120 2 100 50 10 5 5 5 2.5 8100 150 117.5 1.75 100 50 10 5 5 5 2.5 7100 150 115 1.5 100 50 10 5 5 5 2.5 6100 150 112.5 1.25 100 50 10 5 5 5 2.5 5100 150 110 1 10 5 1 0.5 1 1 1 10100 150 109 0.9 1 0.5 0.1 0.05 1 1 1 9100 150 108 0.8 1 0.5 0.1 0.05 1 1 1 8100 150 107 0.7 1 0.5 0.1 0.05 0.5 1 1 7100 150 106 0.6 1 0.5 0.1 0.05 0.5 1 1 6100 150 105 0.5 1 0.5 0.1 0.05 0.5 0.5 0.5 10

122Appen

dix

B.Bisection

Gap

Calcu

lationAlgorith

mTests

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

2 110 140 125 1.5 100 50 10 5 5 5 2.5 6110 140 122.5 1.25 100 50 10 5 5 5 2.5 5110 140 120 1 10 5 1 0.5 1 1 1 10110 140 119 0.9 1 0.5 0.1 0.05 1 1 1 9110 140 118 0.8 1 0.5 0.1 0.05 1 1 1 8110 140 117 0.7 1 0.5 0.1 0.05 0.5 1 1 7110 140 116 0.6 1 0.5 0.1 0.05 0.5 1 1 6110 140 115 0.5 1 0.5 0.1 0.05 0.5 0.5 0.5 10

3 100 140 120 2 100 50 10 5 5 5 2.5 8100 140 117.5 1.75 100 50 10 5 5 5 2.5 7100 140 115 1.5 100 50 10 5 5 5 2.5 6100 140 112.5 1.25 100 50 10 5 5 5 2.5 5100 140 110 1 10 5 1 0.5 1 1 1 10100 140 109 0.9 1 0.5 0.1 0.05 1 1 1 9

4 100 140 115 1.5 100 50 10 5 5 5 2.5 6100 140 112.5 1.25 100 50 10 5 5 5 2.5 5100 140 110 1 10 5 1 0.5 1 1 1 10100 140 109 0.9 1 0.5 0.1 0.05 1 1 1 9100 140 108 0.8 1 0.5 0.1 0.05 1 1 1 8

5 1000 1500 1250 25 1000 500 100 50 50 50 50 51000 1500 1200 20 1000 500 100 50 50 50 25 81000 1500 1175 17.5 1000 500 100 50 50 50 25 71000 1500 1150 15 1000 500 100 50 50 50 25 61000 1500 1125 12.5 1000 500 100 50 50 50 25 51000 1500 1100 10 100 50 10 5 10 10 10 10

123

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

1000 1500 1090 9 100 50 10 5 10 10 10 91000 1500 1080 8 100 50 10 5 10 10 10 81000 1500 1070 7 100 50 10 5 5 10 10 71000 1500 1060 6 100 50 10 5 5 10 10 61000 1500 1050 5 100 50 10 5 5 5 5 101000 1500 1045 4.5 100 50 10 5 5 5 5 91000 1500 1040 4 100 50 10 5 5 5 5 81000 1500 1035 3.5 100 50 10 5 5 5 5 7

6 1000 1200 1100 10 100 50 10 5 10 10 10 101000 1200 1090 9 100 50 10 5 10 10 10 91000 1200 1080 8 100 50 10 5 10 10 10 81000 1200 1070 7 100 50 10 5 5 10 10 71000 1200 1060 6 100 50 10 5 5 10 10 61000 1200 1050 5 100 50 10 5 5 5 5 101000 1200 1045 4.5 100 50 10 5 5 5 5 91000 1200 1040 4 100 50 10 5 5 5 5 81000 1200 1035 3.5 100 50 10 5 5 5 5 71000 1200 1030 3 100 50 10 5 5 5 5 61000 1200 1025 2.5 100 50 10 5 5 5 5 5

7 12000 15000 13500 150 10000 5000 1000 500 500 500 250 612000 15000 13250 125 10000 5000 1000 500 500 500 250 512000 15000 13000 100 1000 500 100 50 100 100 100 1012000 15000 12900 90 1000 500 100 50 100 100 100 912000 15000 12800 80 1000 500 100 50 100 100 100 812000 15000 12700 70 1000 500 100 50 50 100 100 7

124Appen

dix

B.Bisection

Gap

Calcu

lationAlgorith

mTests

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

12000 15000 12600 60 1000 500 100 50 50 100 100 612000 15000 12500 50 1000 500 100 50 50 50 50 1012000 15000 12450 45 1000 500 100 50 50 50 50 912000 15000 12400 40 1000 500 100 50 50 50 50 812000 15000 12350 35 1000 500 100 50 50 50 50 712000 15000 12300 30 1000 500 100 50 50 50 50 612000 15000 12250 25 1000 500 100 50 50 50 50 512000 15000 12200 20 1000 500 100 50 50 50 25 812000 15000 12175 17.5 1000 500 100 50 50 50 25 712000 15000 12150 15 1000 500 100 50 50 50 25 612000 15000 12125 12.5 1000 500 100 50 50 50 25 512000 15000 12100 10 100 50 10 5 10 10 10 1012000 15000 12090 9 100 50 10 5 10 10 10 9

8 0 160000 80000 8000 100000 50000 10000 5000 10000 10000 10000 80 160000 70000 7000 100000 50000 10000 5000 5000 10000 10000 70 160000 60000 6000 100000 50000 10000 5000 5000 10000 10000 60 160000 50000 5000 100000 50000 10000 5000 5000 5000 5000 100 160000 45000 4500 100000 50000 10000 5000 5000 5000 5000 90 160000 40000 4000 100000 50000 10000 5000 5000 5000 5000 80 160000 35000 3500 100000 50000 10000 5000 5000 5000 5000 70 160000 30000 3000 100000 50000 10000 5000 5000 5000 5000 60 160000 25000 2500 100000 50000 10000 5000 5000 5000 5000 50 160000 20000 2000 100000 50000 10000 5000 5000 5000 2500 80 160000 17500 1750 100000 50000 10000 5000 5000 5000 2500 70 160000 15000 1500 100000 50000 10000 5000 5000 5000 2500 6

125

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

0 160000 12500 1250 100000 50000 10000 5000 5000 5000 2500 50 160000 10000 1000 10000 5000 1000 500 1000 1000 1000 100 160000 9000 900 10000 5000 1000 500 1000 1000 1000 90 160000 8000 800 10000 5000 1000 500 1000 1000 1000 80 160000 7000 700 10000 5000 1000 500 500 1000 1000 70 160000 6000 600 10000 5000 1000 500 500 1000 1000 60 160000 5000 500 10000 5000 1000 500 500 500 500 100 160000 4500 450 10000 5000 1000 500 500 500 500 90 160000 4000 400 10000 5000 1000 500 500 500 500 80 160000 3500 350 10000 5000 1000 500 500 500 500 70 160000 3000 300 10000 5000 1000 500 500 500 500 60 160000 2500 250 10000 5000 1000 500 500 500 500 50 160000 2000 200 10000 5000 1000 500 500 500 250 80 160000 1750 175 10000 5000 1000 500 500 500 250 70 160000 1500 150 10000 5000 1000 500 500 500 250 60 160000 1250 125 10000 5000 1000 500 500 500 250 50 160000 1000 100 1000 500 100 50 100 100 100 100 160000 900 90 1000 500 100 50 100 100 100 90 160000 800 80 1000 500 100 50 100 100 100 80 160000 700 70 1000 500 100 50 50 100 100 70 160000 600 60 1000 500 100 50 50 100 100 60 160000 500 50 1000 500 100 50 50 50 50 100 160000 450 45 1000 500 100 50 50 50 50 90 160000 400 40 1000 500 100 50 50 50 50 80 160000 350 35 1000 500 100 50 50 50 50 7

126Appen

dix

B.Bisection

Gap

Calcu

lationAlgorith

mTests

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

0 160000 300 30 1000 500 100 50 50 50 50 60 160000 250 25 1000 500 100 50 50 50 50 50 160000 200 20 1000 500 100 50 50 50 25 80 160000 175 17.5 1000 500 100 50 50 50 25 70 160000 150 15 1000 500 100 50 50 50 25 60 160000 125 12.5 1000 500 100 50 50 50 25 50 160000 100 10 100 50 10 5 10 10 10 100 160000 90 9 100 50 10 5 10 10 10 90 160000 80 8 100 50 10 5 10 10 10 80 160000 70 7 100 50 10 5 5 10 10 70 160000 60 6 100 50 10 5 5 10 10 60 160000 50 5 100 50 10 5 5 5 5 100 160000 45 4.5 100 50 10 5 5 5 5 90 160000 40 4 100 50 10 5 5 5 5 80 160000 35 3.5 100 50 10 5 5 5 5 70 160000 30 3 100 50 10 5 5 5 5 60 160000 25 2.5 100 50 10 5 5 5 5 50 160000 20 2 100 50 10 5 5 5 2.5 80 160000 17.5 1.75 100 50 10 5 5 5 2.5 70 160000 15 1.5 100 50 10 5 5 5 2.5 60 160000 12.5 1.25 100 50 10 5 5 5 2.5 50 160000 10 1 10 5 1 0.5 1 1 1 100 160000 9 0.9 1 0.5 0.1 0.05 1 1 1 90 160000 8 0.8 1 0.5 0.1 0.05 1 1 1 80 160000 7 0.7 1 0.5 0.1 0.05 0.5 1 1 7

127

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

9 0.01 0.09 0.05 0.004 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 80.01 0.09 0.045 0.0035 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 70.01 0.09 0.04 0.003 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 60.01 0.09 0.035 0.0025 0.01 0.005 0.001 0.0005 0.001 0.004 0.004 70.01 0.09 0.031 0.0021 0.01 0.005 0.001 0.0005 0.001 0.004 0.004 60.01 0.09 0.027 0.0017 0.01 0.005 0.001 0.0005 0.001 0.002 0.002 90.01 0.09 0.025 0.0015 0.01 0.005 0.001 0.0005 0.001 0.002 0.002 80.01 0.09 0.023 0.0013 0.01 0.005 0.001 0.0005 0.001 0.002 0.002 70.01 0.09 0.021 0.0011 0.01 0.005 0.001 0.0005 0.001 0.002 0.002 6

10 0.01 0.5 0.255 0.0245 0.1 0.05 0.01 0.005 0.01 0.04 0.04 70.01 0.5 0.215 0.0205 0.1 0.05 0.01 0.005 0.01 0.04 0.04 60.01 0.5 0.175 0.0165 0.1 0.05 0.01 0.005 0.01 0.02 0.02 90.01 0.5 0.155 0.0145 0.1 0.05 0.01 0.005 0.01 0.02 0.02 80.01 0.5 0.135 0.0125 0.1 0.05 0.01 0.005 0.01 0.02 0.02 70.01 0.5 0.115 0.0105 0.1 0.05 0.01 0.005 0.01 0.02 0.02 60.01 0.5 0.095 0.0085 0.01 0.005 0.001 0.0005 0.01 0.01 0.01 90.01 0.5 0.085 0.0075 0.01 0.005 0.001 0.0005 0.01 0.01 0.01 80.01 0.5 0.075 0.0065 0.01 0.005 0.001 0.0005 0.005 0.01 0.01 70.01 0.5 0.065 0.0055 0.01 0.005 0.001 0.0005 0.005 0.01 0.01 60.01 0.5 0.055 0.0045 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 90.01 0.5 0.05 0.004 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 80.01 0.5 0.045 0.0035 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 70.01 0.5 0.04 0.003 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 60.01 0.5 0.035 0.0025 0.01 0.005 0.001 0.0005 0.001 0.004 0.004 70.01 0.5 0.031 0.0021 0.01 0.005 0.001 0.0005 0.001 0.004 0.004 6

128Appen

dix

B.Bisection

Gap

Calcu

lationAlgorith

mTests

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

0.01 0.5 0.027 0.0017 0.01 0.005 0.001 0.0005 0.001 0.002 0.002 911 0 1 0.5 0.05 0.1 0.05 0.01 0.005 0.05 0.05 0.05 10

0 1 0.45 0.045 0.1 0.05 0.01 0.005 0.05 0.05 0.05 90 1 0.4 0.04 0.1 0.05 0.01 0.005 0.05 0.05 0.05 80 1 0.35 0.035 0.1 0.05 0.01 0.005 0.05 0.05 0.05 70 1 0.3 0.03 0.1 0.05 0.01 0.005 0.05 0.05 0.05 60 1 0.25 0.025 0.1 0.05 0.01 0.005 0.01 0.04 0.04 70 1 0.21 0.021 0.1 0.05 0.01 0.005 0.01 0.04 0.04 60 1 0.17 0.017 0.1 0.05 0.01 0.005 0.01 0.02 0.02 90 1 0.15 0.015 0.1 0.05 0.01 0.005 0.01 0.02 0.02 80 1 0.13 0.013 0.1 0.05 0.01 0.005 0.01 0.02 0.02 70 1 0.11 0.011 0.1 0.05 0.01 0.005 0.01 0.02 0.02 60 1 0.09 0.009 0.01 0.005 0.001 0.0005 0.01 0.01 0.01 100 1 0.08 0.008 0.01 0.005 0.001 0.0005 0.01 0.01 0.01 90 1 0.07 0.007 0.01 0.005 0.001 0.0005 0.005 0.01 0.01 80 1 0.06 0.006 0.01 0.005 0.001 0.0005 0.005 0.01 0.01 70 1 0.05 0.005 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 10

12 0 1000000 500000 50000 1000000 500000 100000 50000 50000 50000 50000 100 1000000 450000 45000 1000000 500000 100000 50000 50000 50000 50000 90 1000000 400000 40000 1000000 500000 100000 50000 50000 50000 50000 80 1000000 350000 35000 1000000 500000 100000 50000 50000 50000 50000 70 1000000 300000 30000 1000000 500000 100000 50000 50000 50000 50000 60 1000000 250000 25000 1000000 500000 100000 50000 50000 50000 50000 50 1000000 200000 20000 1000000 500000 100000 50000 50000 50000 25000 80 1000000 175000 17500 1000000 500000 100000 50000 50000 50000 25000 7

129

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

0 1000000 150000 15000 1000000 500000 100000 50000 50000 50000 25000 60 1000000 125000 12500 1000000 500000 100000 50000 50000 50000 25000 50 1000000 100000 10000 100000 50000 10000 5000 10000 10000 10000 100 1000000 90000 9000 100000 50000 10000 5000 10000 10000 10000 90 1000000 80000 8000 100000 50000 10000 5000 10000 10000 10000 80 1000000 70000 7000 100000 50000 10000 5000 5000 10000 10000 70 1000000 60000 6000 100000 50000 10000 5000 5000 10000 10000 60 1000000 50000 5000 100000 50000 10000 5000 5000 5000 5000 100 1000000 45000 4500 100000 50000 10000 5000 5000 5000 5000 90 1000000 40000 4000 100000 50000 10000 5000 5000 5000 5000 80 1000000 35000 3500 100000 50000 10000 5000 5000 5000 5000 70 1000000 30000 3000 100000 50000 10000 5000 5000 5000 5000 60 1000000 25000 2500 100000 50000 10000 5000 5000 5000 5000 50 1000000 20000 2000 100000 50000 10000 5000 5000 5000 2500 80 1000000 17500 1750 100000 50000 10000 5000 5000 5000 2500 70 1000000 15000 1500 100000 50000 10000 5000 5000 5000 2500 60 1000000 12500 1250 100000 50000 10000 5000 5000 5000 2500 50 1000000 10000 1000 10000 5000 1000 500 1000 1000 1000 100 1000000 9000 900 10000 5000 1000 500 1000 1000 1000 90 1000000 8000 800 10000 5000 1000 500 1000 1000 1000 80 1000000 7000 700 10000 5000 1000 500 500 1000 1000 70 1000000 6000 600 10000 5000 1000 500 500 1000 1000 60 1000000 5000 500 10000 5000 1000 500 500 500 500 100 1000000 4500 450 10000 5000 1000 500 500 500 500 90 1000000 4000 400 10000 5000 1000 500 500 500 500 8

130Appen

dix

B.Bisection

Gap

Calcu

lationAlgorith

mTests

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

0 1000000 3500 350 10000 5000 1000 500 500 500 500 70 1000000 3000 300 10000 5000 1000 500 500 500 500 60 1000000 2500 250 10000 5000 1000 500 500 500 500 50 1000000 2000 200 10000 5000 1000 500 500 500 250 80 1000000 1750 175 10000 5000 1000 500 500 500 250 70 1000000 1500 150 10000 5000 1000 500 500 500 250 60 1000000 1250 125 10000 5000 1000 500 500 500 250 50 1000000 1000 100 1000 500 100 50 100 100 100 100 1000000 900 90 1000 500 100 50 100 100 100 90 1000000 800 80 1000 500 100 50 100 100 100 80 1000000 700 70 1000 500 100 50 50 100 100 70 1000000 600 60 1000 500 100 50 50 100 100 60 1000000 500 50 1000 500 100 50 50 50 50 100 1000000 450 45 1000 500 100 50 50 50 50 90 1000000 400 40 1000 500 100 50 50 50 50 80 1000000 350 35 1000 500 100 50 50 50 50 70 1000000 300 30 1000 500 100 50 50 50 50 60 1000000 250 25 1000 500 100 50 50 50 50 50 1000000 200 20 1000 500 100 50 50 50 25 80 1000000 175 17.5 1000 500 100 50 50 50 25 70 1000000 150 15 1000 500 100 50 50 50 25 60 1000000 125 12.5 1000 500 100 50 50 50 25 50 1000000 100 10 100 50 10 5 10 10 10 100 1000000 90 9 100 50 10 5 10 10 10 90 1000000 80 8 100 50 10 5 10 10 10 8

131

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

0 1000000 70 7 100 50 10 5 5 10 10 70 1000000 60 6 100 50 10 5 5 10 10 60 1000000 50 5 100 50 10 5 5 5 5 100 1000000 45 4.5 100 50 10 5 5 5 5 90 1000000 40 4 100 50 10 5 5 5 5 8

13 -100 0 -50 5 100 50 10 5 5 5 5 10-100 0 -55 4.5 100 50 10 5 5 5 5 9-100 0 -60 4 100 50 10 5 5 5 5 8-100 0 -65 3.5 100 50 10 5 5 5 5 7-100 0 -70 3 100 50 10 5 5 5 5 6-100 0 -75 2.5 100 50 10 5 5 5 5 5-100 0 -80 2 100 50 10 5 5 5 2.5 8-100 0 -82.5 1.75 100 50 10 5 5 5 2.5 7-100 0 -85 1.5 100 50 10 5 5 5 2.5 6-100 0 -87.5 1.25 100 50 10 5 5 5 2.5 5-100 0 -90 1 10 5 1 0.5 1 1 1 10-100 0 -91 0.9 1 0.5 0.1 0.05 1 1 1 9-100 0 -92 0.8 1 0.5 0.1 0.05 1 1 1 8-100 0 -93 0.7 1 0.5 0.1 0.05 0.5 1 1 7

14 -100 -50 -75 2.5 100 50 10 5 5 5 5 5-100 -50 -80 2 100 50 10 5 5 5 2.5 8-100 -50 -82.5 1.75 100 50 10 5 5 5 2.5 7-100 -50 -85 1.5 100 50 10 5 5 5 2.5 6-100 -50 -87.5 1.25 100 50 10 5 5 5 2.5 5-100 -50 -90 1 10 5 1 0.5 1 1 1 10

132Appen

dix

B.Bisection

Gap

Calcu

lationAlgorith

mTests

CaseLowerBound

UpperBound

Assess-mentpoint

Diff. /10


Roun-dedGap

Correc-tedGap

FinalGap

Chunks

-100 -50 -91 0.9 1 0.5 0.1 0.05 1 1 1 9-100 -50 -92 0.8 1 0.5 0.1 0.05 1 1 1 8-100 -50 -93 0.7 1 0.5 0.1 0.05 0.5 1 1 7-100 -50 -94 0.6 1 0.5 0.1 0.05 0.5 1 1 6-100 -50 -95 0.5 1 0.5 0.1 0.05 0.5 0.5 0.5 10

15 -0.2 -0.1 -0.15 0.005 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 10-0.2 -0.1 -0.155 0.0045 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 9-0.2 -0.1 -0.16 0.004 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 8-0.2 -0.1 -0.165 0.0035 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 7-0.2 -0.1 -0.17 0.003 0.01 0.005 0.001 0.0005 0.005 0.005 0.005 6-0.2 -0.1 -0.175 0.0025 0.01 0.005 0.001 0.0005 0.001 0.004 0.004 7

Table B.1: Bisection Algorithm Tests

Bibliography

[Ariely 2010] D. Ariely. Predictably irrational, revised and expanded edition: The hiddenforces that shape our decisions. Business & economics. HarperCollins, 2010. (Citedon page 67.)

[Borges 1944] Jorge Luis Borges. Ficciones. Editorial Sur, Buenos Aires, Argentina, 1944.(Cited on pages 1 and 2.)

[Cambridge University Press ] Cambridge University Press. Cambridge Business EnglishDictionary. Downloaded from http://dictionary.cambridge.org/dictionary/business-english/moral-hazard. Accessed on 2015-07-12. (Cited on page 3.)

[Cogentus Consulting Ltd ] Cogentus Consulting Ltd. Overview of Techniques (“Deci-sion Smart” Software). Downloaded from http://www.cogentus.co.uk/support/decision-making/overview-of-techniques. Accessed on 2015-07-19. (Cited onpage 35.)

[Dias 2004] L.C. Dias and A. Tsoukiàs. On the constructive and other approaches inDecision Aiding. In C.A Hengeller Antunes, J. Figueira and J. Clímaco, editeurs,Proceedings of the 56th meeting of the EURO MCDA working group, pages 13–28.CCDRC, Coimbra, 2004. (Cited on pages 8 and 10.)

[Donegan 1991] H.A. Donegan and F.J. Dodd. A note on Saaty’s random indexes. Math-ematical and Computer Modelling, vol. 15, no. 10, pages 135 – 137, 1991. (Citedon page 114.)

[EURO Working Group on Multicriteria Decision Aiding ] EURO Working Group onMulticriteria Decision Aiding. MCDA Software List. Downloaded from http://www.cs.put.poznan.pl/ewgmcda/index.php/software. Accessed on 2015-07-13. (Cited on pages 22 and 30.)

[Figueira 2002] Jose Figueira and Bernard Roy. Determining the weights of criteria inthe ELECTRE type methods with a revised Simos’ procedure. European Journal ofOperational Research, vol. 139, no. 2, pages 317–326, 2002. (Cited on page 91.)

[Figueira 2005a] J. Figueira, S. Greco and M. Ehrgott. Multiple criteria decision analysis:State of the art surveys. Springer Verlag, Boston, Dordrecht, London, 2005. (Citedon pages 8, 12 and 92.)

[Figueira 2005b] J. Figueira, V. Mousseau and B. Roy. ELECTRE methods. In J. Figueira,S. Greco and M. Ehrgott, editeurs, Multiple Criteria Decision Analysis: State ofthe Art Surveys, pages 133–162. Springer Verlag, Boston, Dordrecht, London, 2005.(Cited on page 12.)

[Fishburn 1967] P.C. Fishburn. Methods of estimating additive utilities. ManagementScience, vol. 13, no. 7, 1967. (Cited on page 13.)

http://dictionary.cambridge.org/dictionary/business-english/moral-hazard

http://dictionary.cambridge.org/dictionary/business-english/moral-hazard

http://www.cogentus.co.uk/support/decision-making/overview-of-techniques

http://www.cogentus.co.uk/support/decision-making/overview-of-techniques

http://www.cs.put.poznan.pl/ewgmcda/index.php/software

http://www.cs.put.poznan.pl/ewgmcda/index.php/software

134 Bibliography

[Guitouni 1998] Adel Guitouni and Jean-Marc Martel. Tentative guidelines to help choos-ing an appropriate {MCDA} method. European Journal of Operational Research,vol. 109, no. 2, pages 501 – 521, 1998. (Cited on page 13.)

[Keeney 1993] R.L. Keeney and H. Raiffa. Decisions with multiple objectives: Prefer-ences and value trade-offs. Wiley series in probability and mathematical statistics.Applied probability and statistics. Cambridge University Press, 1993. (Cited onpage 13.)

[L’Agence pour la Création d’Entreprises 2015] L’Agence pour la Création d’Entreprises.L’Agence pour la Création d’Entreprises. Downloaded from http://www.apce.com/pid1627/comparaison-rapide.html?espace=1&tp=1, 2015. Accessed on2015-03-15. (Cited on page 50.)

[Maslow 1966] Abraham H. Maslow. The Psychology of science: A Reconnaissance.Harper and Row, New York, United States of America, 1966. (Cited on page 21.)

[Miettinen 2006] Kaisa Miettinen and Marko M. Mäkelä. Synchronous approach in in-teractive multiobjective optimization. European Journal of Operational Research,vol. 170, no. 3, pages 909 – 922, 2006. (Cited on page 29.)

[Munda 2004] Giuseppe Munda. Social multi-criteria evaluation: Methodological founda-tions and operational consequences. European Journal of Operational Research,vol. 158, no. 3, pages 662 – 677, 2004. (Cited on page 3.)

[Pavesi 2000] P.F.J. Pavesi, P. Bonatti, D. Avenburg and M. Aguirre. La decisión: Suteoría y práctica. Norma, Buenos Aires, Argentina, 2000. (Cited on pages 1, 2and 3.)

[Project Management Institute 2013] Project Management Institute. A Guide To TheProject Management Body Of Knowledge (PMBOK(R) Guide). Project Manage-ment Institute, 5th édition, 2013. (Cited on page 47.)

[Roy 1990] B. Roy. The outranking approach and the foundations of ELECTRE methods.In C. Bana e Costa, editeur, Readings in Multiple Criteria Decision Aid, pages155–183. Springer-Verlag, Berlin, 1990. (Cited on page 12.)

[Roy 2002] B. Roy and J. Figueira. Determining the weights of criteria in the ELECTREtype methods with a revised Simos’ procedure. European Journal of OperationalResearch, vol. 139, pages 317–326, 2002. (Cited on page 46.)

[Roy 2005] Bernard Roy. Paradigms and Challenges. In Multiple Criteria Decision Anal-ysis: State of the Art Surveys, volume 78, pages 3–24. Springer New York, 2005.(Cited on page 7.)

[Roy 2013] Bernard Roy and Roman Slowinski. Questions guiding the choice of a mul-ticriteria decision aiding method. EURO Journal on Decision Processes, vol. 1,no. 1-2, pages 69–97, 2013. (Cited on pages 13, 14, 15 and 16.)

http://www.apce.com/pid1627/comparaison-rapide.html?espace=1&tp=1

http://www.apce.com/pid1627/comparaison-rapide.html?espace=1&tp=1

Bibliography 135

[Saaty 2005] ThomasL. Saaty. The Analytic Hierarchy and Analytic Network Processesfor the Measurement of Intangible Criteria and for Decision-Making. In MultipleCriteria Decision Analysis: State of the Art Surveys, volume 78, pages 345–405.Springer New York, 2005. (Cited on page 11.)

[Santiteerakul 2012] S. Santiteerakul, A. Sekhari and A. Bouras. Selecting Multiple Cri-teria Decision Aid for Measuring Sustainability Supply Chain Performance. InInternational Workshop on Green Supply Chain, Arras, France, June 2012. (Citedon page 8.)

[Shackle 1966] George L. S. Shackle. Decisión, orden y tiempo en las actividades humanas.Tecnos, Madrid, Spain, 1966. (Cited on page 1.)

[Tzu 1988] Sun Tzu. The Art of War. Shambhala Publications, 1988. (Cited on page 1.)

[van Herwijnen 2010] Marjan van Herwijnen. Multi-Attribute Value Theory (MAVT).Downloaded from http://www.ivm.vu.nl/en/images/mca1_tcm53-161527.pdf,2010. Accessed on 2015-07-15. (Cited on page 12.)

[von Winterfeldt 1986] Detlof von Winterfeldt and Ward Edwards. Decision analysis andbehavioral research. Cambridge University Press, 1986. (Cited on pages 2, 3 and 4.)

http://www.ivm.vu.nl/en/images/mca1_tcm53-161527.pdf

Abstract:

To start with, this Thesis has the objective of further elaborating the ideaspresented in the state of the art with regards to method choice in Multi-CriteriaDecision Analysis (MCDA).

Secondly, on the basis of the aforementioned theoretical contribution and, after identifyingand considering the advantages and disadvantages present in non-commercial andcommercial decision applications, a new multi-method MCDA software applicationwill be delivered.

Furthermore, the project intends to democratise the access to decision aiding.While this undoubtedly delivers a social contribution, it might also lead the creationof a new market —the “retail” decision-aiding market—, whose potential profitabilityis worth being analysed.

Keywords: decision-aiding, software, multi-method

master thesis - it4bi · master thesis maximiliano ariel lópez leveraging decision aiding and...

Documents