aircraft design thesis

A Probabilistic Multi-Criteria Decision Making

Technique for Conceptual and Preliminary

Aerospace Systems Design

A Thesis

presented to

The Academic Faculty

by

Oliver Bandte

In Partial Fulfillment

of the Requirements for the Degree

Doctor of Philosophy in Aerospace Engineering

Georgia Institute of Technology

September 2000

Copyright © 2000 by Oliver Bandte

Oliver Bandte ii

A Probabilistic Multi-Criteria Decision Making Technique for

Conceptual and Preliminary Aerospace Systems Design

Approved:

_________________________________Professor Dimitri Mavris, Advisor

_________________________________Professor Daniel Schrage, Advisor

_________________________________Professor Leonid Bunimovich

Date Approved ____________________

_________________________________Professor James Craig

_________________________________Professor Christian Houdré

Oliver Bandte iii

ACKNOWLEDGEMENTS

The hardest part of a dissertation is finding the proper words of appreciation for

all the people that where involved in its creation process. There are my advisors,

Dr. Schrage, without whom I would not be at Georgia Tech in the first place, and

Dr. Mavris, who convinced me of staying on for the Master and Ph.D. after my German

thesis. They truly made my work at the Aerospace Systems Design Laboratory an

unforgettable experience I will benefit from for life. There is the thesis committee with

Dr. Bunimovich and Dr. Houdré, whose patience and strong will to meet me half way on

the bridge between engineering and mathematics can not be commended enough, as well

as Dr. Craig who had the vision to bring probabilistic techniques to fields of engineering

other than aerospace. Of course, great contributions were made by almost all students of

ASDL, with a special thanks to everybody that was answering my nagging computer

related questions again and again and again.

A very special thanks has to go to my parents, who have always stood by my side

and supported me one hundred percent. Particularly, the last three years have been the

hardest they had to suffer through, without me being able to give back as much as they

deserved. Included in this appreciation is of course my brother without whom I simply

would not have been able to finish my thesis after my dad’s stroke.

Last but not least, my dearest love and thanks to Joanna (and her family for the

recent months). Without your presence and love, my U.S. experience would have been

less colorful. You challenged and provided the balance in my life that carried me through

times of much uncertainty. I am more than looking forward to spending the rest of my

life with you.

Thank you all!

OB

August 2000

Atlanta, GA.

Oliver Bandte iv

TABLE OF CONTENTS

ACKNOWLEDGEMENTS _______________________________________________iii

TABLE OF CONTENTS _________________________________________________iv

LIST OF FIGURES_____________________________________________________ vii

LIST OF TABLES _____________________________________________________ xii

NOMENCLATURE ____________________________________________________xiii

SUMMARY __________________________________________________________xvi

CHAPTER I - INTRODUCTION___________________________________________ 1

Systems Engineering _______________________________________________ 1

Systems Design ___________________________________________________ 4

Uncertainty in Systems Design ______________________________________ 13

Research Quest __________________________________________________ 20

CHAPTER II - DETERMINISTIC MULTI-CRITERIA DECISION MAKING

TECHNIQUES _____________________________________________________ 24

Product Selection_________________________________________________ 26

Optimization ____________________________________________________ 34

CHAPTER III - PROBABILISTIC DESIGN METHODS_______________________ 49

Metamodel/Monte-Carlo Simulation _________________________________ 52

Fast Probability Integration_________________________________________ 55

Oliver Bandte v

Illustrative Example ______________________________________________ 60

CHAPTER IV - MULTI-VARIATE PROBABILITY THEORY _________________ 66

CHAPTER V - A NEW PROBABILISTIC MULTI-CRITERIA DECISION MAKING

TECHNIQUE ______________________________________________________ 71

Introduction _____________________________________________________ 71

Algorithms______________________________________________________ 74

Five Schemes for Implementation____________________________________ 85

Joint Probabilistic Decision Making Technique _________________________ 98

Optimization ___________________________________________________ 103

Product Selection________________________________________________ 112

Requirement Trade-Off ___________________________________________ 118

CHAPTER VI - IMPLEMENTATION INTO SYSTEMS DESIGN ______________ 122

Feasibility Problem ______________________________________________ 122

Optimization of a Supersonic Commercial Transport____________________ 131

Product Selection________________________________________________ 140

Requirements Trade-Off Analysis and Discussion ______________________ 150

Example Equation System with Ten Criteria __________________________ 158

CHAPTER VII - CONCLUSIONS________________________________________ 166

Joint Probabilistic Decision Making Technique in Design________________ 166

Algorithms for Determining the Joint Probability Distribution ____________ 167

JPDM for Optimization___________________________________________ 169

JPDM for Product Selection _______________________________________ 171

Oliver Bandte vi

Research Questions and Answers ___________________________________ 173

Recommendations _______________________________________________ 174

APPENDIX A - DISTRIBUTIONS _______________________________________ 177

APPENDIX B - CORRELATION FUNCTIONS ____________________________ 179

APPENDIX C - NUMBER OF SAMPLES FOR THE MONTE-CARLO

SIMULATION ____________________________________________________ 184

APPENDIX D - JOINT PROBABILISTIC DECISION MAKING TECHNIQUE

- THE COMPUTER PROGRAM -_____________________________________ 187

List of Files ____________________________________________________ 189

BIBLIOGRAPHY _____________________________________________________ 209

VITA _______________________________________________________________ 220

Oliver Bandte vii

LIST OF FIGURES

Figure 1.1: Design As a Decision Making Process 21

Figure 2.1: Decision Tree for MADM Technique Selection 28

Figure 2.2: Euclidean Distance to the Ideal and Negative-Ideal Solutions 33

Figure 2.3: Decision Tree for MODM Technique Selection 35

Figure 2.4: Parametric Method of Two Criteria with a Convex Set 44

Figure 2.5: Parametric Method of Two Criteria with a Nonconvex Set 44

Figure 2.6: Surface Plot for f1and f2 45

Figure 3.1: Probabilistic Design Method #1 50



Figure 3.4: Face Centered Central Composite Design 54

Figure 3.5: Objective Function Contours 56

Figure 3.6: Joint Probability Distribution 56

Figure 3.7: Most Probable Point Location 57

Figure 3.8: Visualization of MPP 57

Figure 3.9: AMV Method 59

Figure 3.10: Change in Optima for f1 and f2 with Changing y1 and y2 61

Figure 3.11: CDF Comparison of Method #1 and Method #3 for F1 63

Figure 3.12: CDF Comparison of Method #1,#2, and #3 for F2 63

Figure 3.13: Response Surface for Equation 3.5 64

Figure 3.14: Response Surface for Equation 3.6 64

Figure 3.15: Response Surface for Equation 3.5 (10,000 Samples) 65

Figure 3.16: Response Surface for Equation 3.5 (1,000 Samples) 65

Oliver Bandte viii

Figure 4.1: Joint and Marginal PDF of Continuous Criteria X and Y 68

Figure 5.1: Filling the Gap in the Design Process 73

Figure 5.2: Applicability of MADM, MODM, and JPDM 74

Figure 5.3: Gamma-Distribution for X 84

Figure 5.4: Weibull-Distribution for Y 84

Figure 5.5: Example Joint Probability Distribution in 3D 84

Figure 5.6: Contour Plot of Example Joint Probability Distribution 84

Figure 5.7: Five Schemes for the Evaluation of the Joint Probability Distribution 85

Figure 5.8: Example Distribution Generated by Scheme #I (3D) 87

Figure 5.9: Example Distribution Generated by Scheme #I (2D) 87

Figure 5.10: Distribution Regression for F1 in Scheme #II 88

Figure 5.11: Distribution Regression for F2 in Scheme #II 88

Figure 5.12: Example Distribution Generated by Scheme #II (3D) 89

Figure 5.13: Example Distribution Generated by Scheme #II (2D) 89

Figure 5.14: Example Distribution Generated by Scheme #III (3D) 91

Figure 5.15: Example Distribution Generated by Scheme #III (2D) 91

Figure 5.16: Distribution Regression for F1 in Scheme #IV 92

Figure 5.17: Distribution Regression for F2 in Scheme #IV 92

Figure 5.18: Example Distribution Generated by Scheme #IV (3D) 93

Figure 5.19: Example Distribution Generated by Scheme #IV (2D) 93

Figure 5.20: Distribution Regression for F2 in Scheme #V 95

Figure 5.21: Example Distribution Generated by Scheme #V (3D) 96

Figure 5.22: Example Distribution Generated by Scheme #V (2D) 96

Figure 5.23: Comparison of Joint Distributions from Schemes #I and #II 97

Figure 5.24: Comparison of Joint Distributions from Schemes #III and #IV 97

Figure 5.25: Comparison of Joint Distributions from Schemes #II, #IV and #V 98

Oliver Bandte ix

Figure 5.26: Display of the Joint Probability Distribution for Two Criteria 99

Figure 5.27: Joint Probabilistic Decision Making Technique 100

Figure 5.28: Shifting the Joint Probability Distribution During Optimization 104

Figure 5.29: Joint Probabilistic Decision Making Technique for Optimization 105

Figure 5.30: POS Surface Plot Over Design Space 109

Figure 5.31: Location of Optima of MaxiMin, OEC, and Goal Attainment Method for

100 Samples 110

Figure 5.32: Location of Optima for Different Probabilistic Multi-Objective Optimization

Methods 110

Figure 5.33: Comparing Joint Probability Distributions for Product Selection 114

Figure 5.34: Joint Probabilistic Decision Making Technique for Product Selection 115

Figure 5.35: Comparison of Alternatives Based on POS with or without Preferencing 118

Figure 5.36: Requirement Trade-Off to Gain POS 120

Figure 5.37: Trade-Off to Tighten Requirement 120

Figure 5.38: Joint Probabilistic Decision Making Technique for Requirement Trade-Offs 121

Figure 6.1: Feasible Space in a Design Space 123

Figure 6.2: Five Steps to Aircraft Design 124

Figure 6.3: Five Steps to Aircraft Design with JPDM 125

Figure 6.4: Supersonic Transport Concept 126

Figure 6.5: Illustration of the Kink Location 128

Figure 6.6: Determination of Feasibility with JPDM 129

Figure 6.7: Joint EDF for Take-Off Field Length and Approach Speed (2D) 130

Figure 6.8: Joint EDF for Take-Off Field Length and Approach Speed (3D) 130

Figure 6.9: Joint Normal Distribution for Take-Off Field Length and Approach Speed (2D) 130

Figure 6.10: Joint Normal Distribution for Take-Off Field Length and Approach Speed (3D) 130

Figure 6.11: Notional Supersonic Transport 131

Oliver Bandte x

Figure 6.12: JPDM as Optimization Process 136

Figure 6.13: Optimization Iteration History of POS and Vapp 137

Figure 6.14: Optimization Iteration History of OEC, POS, and Vapp 138

Figure 6.15: B-747 140

Figure 6.16: B-777 140

Figure 6.17: A340 141

Figure 6.18: Supersonic Transport 141

Figure 6.19: JPDM for Product Selection 142

Figure 6.20: Joint Probability Plot for ROI and TOC 144

Figure 6.21: Joint Probability Plot for REV and TOC 144

Figure 6.22: Magnified Joint Probability Plot for ROI and TOC 145

Figure 6.23: Magnified Joint Probability Plot for REV and TOC 145

Figure 6.24: Joint Probability Plot for REV and TOC with New Area of Interest 149

Figure 6.25: Comparison of POS as a Function of Revenue and Cost Requirements 151

Figure 6.26: Joint Probability Plot for REV and TOC (A340 is best) 152

Figure 6.27: Joint Probability Plot for REV and TOC (B-777 is best) 152

Figure 6.28: Joint Probability Plot for REV and TOC (B-747 is best) 153

Figure 6.29: Joint Probability Plot for REV and TOC (SST is best) 153

Figure 6.30: Sensitivity Plot for Changes in Mean for the B-747 and B-777 157

Figure 6.31: Sensitivity Plot for Changes in Standard Deviation for the B-747 and B-777 157

Figure 6.32: Surfaces of Ten Equations 159

Figure 6.33: Maxima of Ten Equations 159

Figure 6.34: POS Response Surface 162

Figure 6.35: OEC Response Surface 162

Figure 6.36: Optimal Solutions 163

Figure B.1: Surface Plot of Equation 5.18 for Two Variables 179

Oliver Bandte xi

Figure B.2: Surface Plot of Equation B.1 180

Figure B.3: Surface Plot of Equation B.2 180

Figure B.4: Joint Probability Plot for Two Beta-Distributions (ρ = 0) 181

Figure B.5: Joint Probability Plot for Two Beta-Distributions with Correlation

Function B.1 (ρ = 0.09) 181


Function B.1 (ρ = 0.9) 181

Figure B.7: Joint Probability Plot for Beta-Distributions with Correlation

Function B.3 (γ = 4.9) 182


Function B.4 (γ = 4.9) 182


Function B.5 (γ = 4.9) 183


Function B.6 (γ = 4.9) 183

Figure C.1: Number of Samples as a Function of Probability Level and Percent Error 186

Figure D.1: Flow Chart of the Joint Probabilistic Decision Making Technique 187

Oliver Bandte xii

LIST OF TABLES

Table 2.1: Comparison of Three Multi-Criteria Optimization Techniques ________________ 46

Table 2.2: Normalized Comparison of Three Multi-Criteria Optimization Techniques ______ 48

Table 5.1: Comparison of Joint and Univariate Probabilities__________________________ 112

Table 6.1: Description of the Baseline ___________________________________________ 126

Table 6.2: Design Variable Description and Range _________________________________ 128

Table 6.3: Range of Allowable Design Variable Values for Optimization _______________ 132

Table 6.4: Economic Parameter Distributions _____________________________________ 134

Table 6.5: Four Alternatives for Product Selection Problem __________________________ 141

Table 6.6: Noise Variable Distributions __________________________________________ 143

Table 6.7: Summary of POS for each Alternative __________________________________ 146

Table 6.8: Comparison of POS for Different ROImin Values __________________________ 150

Table 6.9: Comparison of POS Values for Different Criterion Preferences_______________ 154

Table 6.10: Old and New Noise Variable Distributions______________________________ 155

Table 6.11: Old and New POS Values and Sensitivities for B-747 and B-777 ____________ 157

Table 6.12: Comparison of Joint and Univariate Probabilities for Different Objective

Functions ____________________________________________________________ 165

Table 7.1: Advantages and Disadvantages of EDF and JPM __________________________ 169

Oliver Bandte xiii

NOMENCLATURE

µ : Mean; expected value of a random variable

ρ Correlation Coefficient; normalized covariance of two random

variables

σ : Standard Deviation; measure of dispersion or variability of random

variable values, determined by their deviation from the mean

σ 2 : Variance of a random variable

AMV: Advanced Mean Value method; FPI technique (see page 55)

CDF: Cumulative Distribution Function

DOE: Design of Experiments; set of evaluation points that a metamodel

is based on

EDF: Empirical Distribution Function; discrete probability distribution

based on random samples

FPI: Fast Probability Integration; computer program; collection of

techniques for efficient evaluation of probability distributions of

system characteristics

JPDM: Joint Probabilistic Decision Making technique; product of this

thesis

JPM: Joint Probability Model; joint probability distribution (product of

this thesis)

Oliver Bandte xiv

MADM: Multi-Attribute Decision Making = product selection

MCDM: Multi-Criteria Decision Making; comprised of MADM and

MODM; decision making on the basis of multiple, often

conflicting, criteria

MCS: Monte-Carlo simulation: technique for generating random numbers

MODM: Multi-Objective Decision Making = optimization

OEC: Overall Evaluation Criterion; utility function technique for product

selection and optimization [OEC = ∑=

⋅N

iii fw

1

)(x ]

PDF: Probability Density Function

POS: Probability of Success; probability of meeting customer

requirements and desirements; objective function for probabilistic

multi-criteria optimization and product selection

RSE: Response Surface Equation

RSM: Response Surface Methodology; linear regression technique for

creating metamodels (RSE)

Attribute: Criterion that describes in part the state of a system

Covariance: Measure of the degree of (linear) interrelationship between values

of two random variables

Correlation: Mathematical correspondence of values of two random variables

Criterion: Standard of judgment to test acceptability

Desirement: Customer supplied want that impacts product design

Oliver Bandte xv

Engineering Design: Process that creates a product, which satisfies a need that was not

(sufficiently) satisfied by already existing products

Joint Distribution: Probability distribution of two or more random variables

Metamodel: Model that approximates a more complex model

Model: Mathematical representation of a system

Model Fidelity: Prediction accuracy of a model

Objective: Customer supplied criterion that forms (in part) the bases for the

decision making process

Optimization: Process of finding the best solution among an infinitely large

(open) set of alternatives

Probabilistic Design: Design process that accounts for uncertainties probabilistically

Product Selection: Process of finding the best solution among a fixed (closed) set of

alternatives

Random Variable: Variable with uncertain value; likelihood of values is described by

a probability distribution

Readiness: Level to which a particular technology or concept is implemented

into the system

Requirement: Customer supplied need that must be fulfilled

System: Group of components, attributes, and relationships needed to

accomplish an objective

Trade-Off: Process of giving up one thing for another

Uncertainty: Inability to predict the future with certainty; three classes:

environmental/operational, fidelity, readiness

Oliver Bandte xvi

SUMMARY

It has always been the intention of systems engineering to invent or produce the

best product possible. Many design techniques have been introduced over the course of

decades that try to fulfill this intention. Unfortunately, no technique has succeeded in

combining multi-criteria decision making with probabilistic design. The design

technique developed in this thesis, the Joint Probabilistic Decision Making (JPDM)

technique, successfully overcomes this deficiency by generating a multivariate

probability distribution that serves in conjunction with a criterion value range of interest

as a universally applicable objective function for multi-criteria optimization and product

selection. This new objective function constitutes a meaningful metric, called Probability

of Success (POS), that allows the customer or designer to make a decision based on the

chance of satisfying the customer’s goals. In order to incorporate a joint probabilistic

formulation into the systems design process, two algorithms are created that allow for an

easy implementation into a numerical design framework: the (multivariate) Empirical

Distribution Function and the Joint Probability Model. The Empirical Distribution

Function estimates the probability that an event occurred by counting how many times it

occurred in a given sample. The Joint Probability Model on the other hand is an

analytical parametric model for the multivariate joint probability. It is comprised of the

product of the univariate criterion distributions, generated by the traditional probabilistic

Oliver Bandte xvii

design process, multiplied with a correlation function that is based on available

correlation information between pairs of random variables.

JPDM is an excellent tool for multi-objective optimization and product selection,

because of its ability to transform disparate objectives into a single figure of merit, the

likelihood of successfully meeting all goals or POS. The advantage of JPDM over other

multi-criteria decision making techniques is that POS constitutes a single optimizable

function or metric that enables a comparison of all alternative solutions on an equal basis.

Hence, POS allows for the use of any standard single-objective optimization technique

available and simplifies a complex multi-criteria selection problem into a simple ordering

problem, where the solution with the highest POS is best. By distinguishing between

controllable and uncontrollable variables in the design process, JPDM can account for the

uncertain values of the uncontrollable variables that are inherent to the design problem,

while facilitating an easy adjustment of the controllable ones to achieve the highest

possible POS. Finally, JPDM’s superiority over current multi-criteria decision making

techniques is demonstrated with an optimization of a supersonic transport concept and ten

contrived equations as well as a product selection example, determining an airline’s best

choice among Boeing’s B-747, B-777, Airbus’ A340, and a Supersonic Transport. The

optimization examples demonstrate JPDM’s ability to produce a better solution with a

higher POS than an Overall Evaluation Criterion or Goal Programming approach.

Similarly, the product selection example demonstrates JPDM’s ability to produce a better

solution with a higher POS and different ranking than the Overall Evaluation Criterion or

Technique for Order Preferences by Similarity to the Ideal Solution (TOPSIS) approach.

Oliver Bandte Chapter I 1

CHAPTER I

INTRODUCTION

The history of creating tools and objects that people need is as long as the history

of humanity itself. Man has always striven to better his position. For many situations,

this meant the invention of new tools or devices that helped to overcome obstacles people

were facing. In each case, however, there was an initial need identified that led to the

invention and production of the new device. If those needs are of material nature, it is

fair to say that engineers today typically answer those needs by designing products or

designing processes that produce products. Beakley and Leach point out that, in general,

engineers do the things required to serve the needs of the people and their culture. Their

job is to take knowledge and make practical use of it and “in doing so provide for man’s

material needs and well being.”[Beakley, Leach, 1972] Thus, engineering design can be

described as a process that creates a product, which satisfies a need that was not

(sufficiently) satisfied by already existing products.

Systems Engineering

However, the satisfaction of a particular need often produced new ones. Recent

decades have seen material needs that are so complex that traditional engineering design

methods are no longer suited to yield products that satisfy these needs. These products

are not simple devices that perform a single function, but rather complex systems that


fulfill a number of tasks. The term ‘system’ stems from the Greek word ‘systema,’ which

means ‘organized whole’. With the engineering process in mind, “a system is, by

definition, a group of activities or components that can be bounded where the bounding

rule is: all relevant interdependencies and interactions must be enclosed.”[Starr, 1963]

Blanchard and Fabrycky refine this definition by stating that “the total system, at

whatever level in the hierarchy, consists of all components, attributes, and relationships

needed to accomplish an objective. Each system has an objective, providing a purpose

for which all system components, attributes, and relationships have been organized.

Constraints placed on the system limit its operation and define the boundary within which

it is intended to operate. Similarly, the system places boundaries and constraints on its

subsystems.”[Blanchard, Fabrycky, 1998]

That the objects of concern in aerospace engineering are systems is quite obvious.

An aircraft (system), for example, is comprised of such components as the wing,

fuselage, engines, empennage, landing gear, and so on. Each of these components can be

divided up into smaller units or components. Thus, they are often referred to as

subsystems, showing all characteristics of a system but at a lower level of hierarchy in the

aircraft system. Therefore, general systems engineering methods are extremely

applicable to aerospace systems. Specifically, the following three system elements can

be defined [Blanchard, Fabrycky, 1998]:

1. Components are the operating parts of a system consisting of input, process, and

output. Each system component may assume a variety of values to describe a

system state as set by some control action and one or more restrictions.


2. Attributes are the properties or discernible manifestations of the components of asystem. These attributes characterize the system.

3. Relationships are the links between components and attributes.

If the system is then to satisfy a material need, as argued by Beakley and Leach,

the process that brings the system about can be called systems engineering.[Beakley,

Leach, 1972] Webster’s defines systems engineering as “a branch of engineering using

especially information theory, computer science, and facts from system-analysis studies

to design integrated operational systems for specific complexes.”[Webster, 1996]

Blanchard adds to this definition the creative aspect of engineering by defining, systems

engineering as “the orderly process of bringing a system into being. A system constitutes

a complex combination of resources (in the form of human beings, materials, equipment,

software, facilities, data, etc.) integrated in such a manner as to fulfill a designated

need.”[Blanchard, 1998] Both definitions recognize the engineered system as “an

assemblage or combination of elements or parts forming a complex or unitary whole,

[while its] interrelated components [work] together toward some common objective or

purpose.”[Blanchard, Fabrycky, 1998] However, “systems engineering per se is not

considered as an engineering discipline in the same context as civil engineering,

mechanical engineering, reliability engineering, or any other design specialty area.

Actually, system engineering involves the efforts pertaining to the overall design and

development process employed in the evolution of a system from the point when a need

is first identified, through production and/or construction and the ultimate installation of

that system for consumer use. The objective is to meet the requirements of the consumer

in an effective and efficient manner.”[Blanchard, 1998]


Systems Design

Blanchard’s remark points out that a part of systems engineering involves the

design of the system. According to Webster’s dictionary, to design is “to plan and carry

out in a skillful way or to form [something] in one’s mind.”[Webster, 1996] Dieter,

however, argues that this definition does not emphasize enough the fact, that to design

something means “to create something that has never been.”[Dieter, 1991] However,

when a design has a particular aim, it “is a purposeful activity directed toward the goal of

fulfilling human needs, particularly those which can be met by the technological factors

of our culture.”[Asimow, 1962] If the satisfaction of this need is the ultimate aim of

design, a process has to be established that guarantees the fulfillment of this goal. This

process is an essential element of design theory, which extends intuition and experience

in (systems) engineering design by “providing a framework for evaluating and extending

design concepts. A design theory would make it possible to answer such questions as: Is

this a good design?; Why is this design better than others?; How many design

parameters (DPs) do I need to satisfy the functional requirements (FRs)?; Shall I

abandon the idea or modify the concept?”[Dieter, 1991]

Decision Making in Systems Design

These questions, on the other hand, constitute the decision making problem in

systems engineering. Webster’s dictionary defines decision making as “the act of making

up one’s mind, judging, or reaching a conclusion about something.”[Webster, 1996] The

connection between decision making and design may not be apparent from this

definition, but when considering the engineer’s aim of product creation, it can be realized


that design, or resource allocation, involves the process of decision making. As

Hazelrigg reasons, a decision is “an irrevocable allocation of resources. The selection of

design parameters for an engineering system such as a computer or an automobile

constitutes an allocation of resources. [Therefore,] design is a decision making process,

and the selections of design parameters represent decisions.”[Hazelrigg 1996]

The close relation between design and decision making can also be seen from

Dixon’s definition: “a decision making problem exists, then, when and only when there is

an objective to be reached, alternative methods of proceeding, and a variety of factors

that are relevant to the evaluation of the alternatives or their probability of

success.”[Dixon, 1966] The ‘objective’ in design is the satisfaction of a need, while

‘alternative methods’ are courses of action that can be taken during the design process.

‘Factors relevant for the evaluation of alternatives’ are the design variables, controlled by

the engineer, that comprise the system and performance parameters by which the system

is evaluated. From these three groups, three elements can be extracted “that concern us

in critical decision making, as it appears in the process, [which] are the alternatives, the

benefits, and the difficulties of implementation.”[Asimow, 1962] While the ‘difficulties

of implementation’ have to be seen in a broader context, comprising economic penalties

as well as technical and environmental constraints.

Defining the ‘objective’ in design somewhat more narrowly, one could say that

the objectives are customer supplied guidelines that form the bases for the decision

making process. This notion links the customer’s need directly to the decision making

process. The importance of this link is also pointed out by Starr: “the decision problem is


to choose that strategy, which best satisfies the decision-maker’s objectives. If the

designer does not know what objectives apply, then only by fortuitous selection can he

succeed.”[Starr, 1963]

This statement clearly shifts the emphasis of decision making towards the

definition and selection of the objectives, which can only be achieved through a needs

analysis. “The needs analysis consists of listing the user needs for the design in brief,

succinct phrases. Each user need should be identified with the basic need it represents.

In addition, there will be technological (performance) needs, time needs, and cost needs.

The needs analysis is not complete until we estimate what resources the user will

exchange for satisfying his needs.”[Dieter, 1991] However, a needs analysis alone is not

enough to supply the engineer with the information required to proceed with the design

process. First, the problem should be defined more precisely to enable the systems

engineer to make educated decisions about the product with respect to the problem that

needs solving. Asimow and Dieter point out the essentials of the problem definition:

“before an attempt is made to find possible solutions for the means of satisfying the need,

the design problem should be identified and formulated. … The information we have

available [for this] comes from the results of the preceding step, [establishing the need,]

particularly the specifications of desired outputs, and from relevant technical knowledge

about environments, resources and general engineering principle.”[Asimow, 1962] And

further, “problem definition is based on identifying the true needs of the user and

formulating them in a set of goals for the problem solution. The problem statement

expresses as specifically as possible what is intended to be accomplished to achieve the


goals. Design specifications are a major component of the problem statement.”[Dieter,

1991]

Consequently, the problem definition yields a set of objectives on which the

engineer can base his design decisions. These parameters are called criteria, described

by Ostrofsky as “the means by which the performance of a system is evaluated. As such,

then, the criteria must emerge from the needs analysis and the problem formulation of the

[systems engineering process].”[Ostrofsky, 1977] These criteria play the essential role in

the decision making process, deeming an alternative solution successful, when their

customer desired levels are met. Asimow states clearly that “as soon as we raise the

problem of finding which of the feasible solutions is best, we are forced to state with

precision the rules by which we are to judge the vague quality of excellence. We must

name and define the attributes, which are to be considered, we must specify how they are

to be measured, and we must establish relative importance. We will refer to such a

composite statement as the design criterion. If the optimization is to be performed

mathematically, these considerations must be set forth in an equation which we shall call

the criterion function.”[Asimow, 1962]

In some traditional approaches to design, these criteria have been termed

design-to parameters, as Blanchard and Fabrycky point out: “evolving from the

operational requirements and [the customer need] is the development of qualitative and

quantitative design-to criteria, [such as system size and weight, range, speed,

performance, capacity, operational availability, reliability, maintainability, supportability,

and cost]. Of particular interest are the quantitative factors or metrics associated with the


system being developed. These metrics, or technical performance measures (TPMs), lead

to the identification of design-dependent parameters (DDPs) and the desired

characteristics that should be incorporated into the design, and must be established

initially as part of the requirements definition process during conceptual

design.”[Blanchard, Fabrycky, 1998] It is important to understand that a ‘design-to-

criterion’ approach to design potentially yields a design that satisfies the specified

criterion exclusively. All other criteria that the system was not specifically designed for

may not be satisfied at all, yielding a lower customer satisfaction and, potentially, a

failure to meet the specified need. Blanchard and Fabrycky even go so far as to claim

that a single criterion system design decision “is the exception rather than the rule.

Multiple criteria considerations arise when both economic and noneconomic elements are

present in the evaluation. In these situations, decision evaluation is facilitated by the use

of a decision evaluation display exhibiting both cost and effectiveness

measures.”[Blanchard, Fabrycky, 1998]

If the decision problem entails several criteria, experience has shown that not all

of them are of equal importance to the customer. Which criterion is more important than

another is often not for the designer to decide, but must be part of the needs analysis

described earlier. This fact is pointed out again by Blanchard and Fabrycky: “some of

these factors may be considered to be more important than others by the customer, which

will, in turn, influence the design process in placing different levels of emphasis on the

selection of design criteria. The result is the identification and prioritization of technical

performance measures (TPMs) for the system overall.”[Blanchard, Fabrycky, 1998] The


importance of preferences among criteria in the decision process should not be

underestimated, since the design outcome depends heavily on them and different

preferences can produce drastically different designs.

Mathematical Models in Systems Design

In order to get an accurate prediction of the system’s operational behavior or state,

it is vital that the criteria for the decision making process are quantifiable metrics or

TPMs, as termed by Blanchard and Fabrycky. Dixon emphasizes this point when he

argues that an engineering design decision question is “one, which can be answered in

terms of parameters that can be calculated or measured. In other words, a question must

be asked that can be answered quantitatively. This is not always easy. It involves

translation of a real physical situation into a paper-and-pencil question. For example, it is

not enough to ask of a system, ‘Will it work?’ Such a question is neither operational nor

specific. Instead, questions must be asked such as: What temperature is ‘too’ hot? What

does ‘quickly’ mean in seconds? An engineering analyst must begin by defining

quantitatively answerable questions.”[Dixon, 1966]

Accommodating the need for quantifiable criteria, analytical methods must be

employed in the design process that yield these criteria. Mathematical models typically

lend themselves best to a systems engineering design approach, since they are frequently

used in engineering, fast in producing the sought for answers, and yield the calculable,

hence quantifiable, outcome desired. Further, Asimow emphasizes the availability and

ease of manipulation of information about the product through the use of mathematical

models. Specifically, he points out that, “as useful as verbal descriptions and graphic


illustration are, symbolic descriptions are uniquely useful, for they can be manipulated

with the facility of mathematical logic in pursuing the implications, which are dormant in

the concept. … The symbolic description becomes a device, which enables the designer

to use information about the concept in order to anticipate analytically the behavior of the

prototype. In this sense, the symbolic description becomes a mathematical archetype of

the physical object which is yet to be materialized.”[Asimow, 1962]

The ease of obtaining answers in the design evaluation processes is particularly

important for systems engineering. The complexity of the product and the related design

processes demands a design method that allows the designer to make informed decisions

at any level in the design process. Blanchard and Fabrycky specifically emphasize the

benefit of mathematical models to systems engineering: “the use of mathematical models

offers significant benefits. … There are many interrelated elements that must be

integrated as a system and not treated on an individual basis. The mathematical model

makes it possible to deal with the problem as an entity and allows consideration of all

major variables of the problem on a simultaneous basis.”[Blanchard, Fabrycky, 1998]

Last but not least, Dieter denotes that design solutions need to be communicated and that

mathematical models are especially helpful here: “engineers use models for thinking,

communications, prediction, control, and training. Since many engineering problems

deal with complex situations, a model often is an aid to visualizing and thinking about the

problem. … Models are vital for communicating, whether via the printed page, the

computer screen, or oral presentation. Generally, we do not really understand a problem

thoroughly until we have predictive ability concerning it. Engineers must make decisions


concerning alternatives. The ability to simulate the operation of a system with a

mathematical model is a great advantage in providing sound information, usually at lower

cost and in less time than if experimentation had been required.”[Dieter, 1991]

Phases in Systems Design

On the other hand, “in the past, the early ‘front-end’ analysis as applied to many

new systems has been minimal.”[Blanchard, Fabrycky, 1998] It is at the ‘front-end’ of

the design process, however, where the requirements are established from the need

analysis. The so called conceptual design phase encompasses the “activities related to

the identification of customer need and the several steps involved in the definition of

system design requirements.”[Blanchard, Fabrycky, 1998] This design phase, where the

basic design concepts are developed, encompasses many critical decisions that drive the

system production and operational characteristics. The decisions are critical, since their

impact is large on the system and changes to the design concept at later stages are

extremely expensive, as Blanchard, Fabrycky, and Ertas point out: “it is at this early stage

in the life cycle (i.e., the conceptual design phase) that major decisions are made relative

to adapting a specific design approach, and it is at this stage that the results of such

decisions can have a great impact on the ultimate characteristics and life-cycle cost of a

system.”[Blanchard, Fabrycky, 1998] “Once the decision to proceed has been made and

funding approval has been granted, it becomes increasingly costly and difficult to make

changes. … The increasing cost of making changes as programs develop is, of course,

due to the sunk cost manpower and materials as well as to the difficulty in reassigning or

terminating an increasingly large number of employees.”[Ertas, 1993] Thus, these


decisions should be as educated as possible to make the best decisions possible. In other

words, it is during the conceptual design phase where mathematical modeling has the

most potential to make a difference in the decision making process.

Employing mathematical models in the conceptual design phase will inevitably

move efforts traditionally associated with preliminary design into the conceptual design

phase. “The purpose of preliminary design is to establish which of the proffered

alternatives is the best design concept. Each of the alternative solutions is subjected to

order of magnitude analyses until the evidence suggests either that the particular solution

is inferior to some of the others, or that it is superior to all of the others. The surviving

solution is tentatively accepted for closer examination. Synthesis studies are initiated for

establishing, to a first approximation, the fineness of the range within which the major

design parameters of the system must be controlled. Further studies investigate the

tolerances in the characteristics of major components and critical materials, which will be

required to insure mutual compatibility and proper fit into the system. Other studies

examine the extent to which perturbations of environmental or internal forces will affect

the stability of the system.”[Asimow, 1962] For aerospace systems specifically, this

means that “during preliminary design the specialists in areas such as structures, landing

gear, and control systems will design and analyze their portion of the aircraft [system].

Testing is initiated in areas such as aerodynamics, propulsion, structures, and stability

and control. A mockup may be constructed at this point.”[Raymer, 1992] With these

descriptions of preliminary and the earlier established intentions of conceptual design, the

use of mathematical models in conceptual as well as preliminary design merges both


phases into one, yielding the ability of the designer to make better, more educated

decisions in satisfying the customer need.

Uncertainty in Systems Design

Despite the use of mathematical models, certain decisions the designer makes will

have to be based on assumptions rather than certain knowledge, due to incomplete

information about the operational environment, availability of new technology, or simply

the uncertainty about the prediction accuracy of the models themselves. The needs

analysis can only describe what kind of qualities the system must or must not have. It

does not provide, in general, the characteristics of the environment in which the system

will operate. This is often left to the designer to determine and assess. However, “there

is usually little assurance that predicted futures will coincide with actual futures. The

physical and economic elements on which a course of action depends may vary from

their estimated values because of chance causes. [On the other hand,] this lack of

certainty about the future makes decision making one of the most challenging tasks faced

by individuals, industry, and government.”[Blanchard, Fabrycky, 1998]

Asimow claims more generally, that “the work of design demands a constant

peering ahead through the curtains of time; for a project started in the present will not be

completed until some time in the future, and the actual product will not be used until an

even more remote time. Two main questions should be asked: one concerns the socio-

economic environment that will exist when the product comes into actual use; the other

refers to the race against technical obsolescence.”[Asimow, 1962] The second question


refers to the problem of uncertainty about the availability of anticipated technological

concepts. Obviously, current technology is readily available for implementation in the

system. However, as Asimow states, it may also be obsolete when the system is actually

fielded. Even worse, certain needs of the customer may not even be met by a system that

is solely built with current technologies. Thus, new technological solutions have to be

found, applied to the components, and incorporated into the system. But these

technological solutions may only be at a conceptual stage in their development, i.e.,

several questions remain concerning their readiness for implementation when needed and

their actual performance level once implemented. On the other hand, technology

readiness is a function of money and time allotted for completion of the technology

development. Asimow points out that “virtually any solution, even a very difficult one,

provided it is physically permissible (e.g., not a perpetual motion machine), can be

carried through to a physically realizable design if a sufficiently large amount of money

is appropriated and an indefinite amount of design time allowed. If the budget of time

and money is limited, then whether or not the design is realizable becomes uncertain.

The greater the limitation, the more uncertainty it raises.”[Asimow, 1962]

The third source of uncertainty, rooted in the prediction accuracy of the models

used in the design process, is often called model fidelity. Dixon summarized this

frequently overlooked form of uncertainty best when he wrote that “engineering models

are not exact replicas of real physical situations. Most nonengineers feel or believe that

engineering is an exact and precise activity. By comparison with the activities of, say,

psychologists, this is quite true. In an absolute sense, however, it is not true at all. The


real physical situations with which engineers work are very complex and are never

analyzed exactly or completely. No one even tries to analyze them exactly or completely.

Assumptions, idealizations, and approximations are expected.”[Dixon, 1966]

In general, uncertainty can be separated into two system specific categories. Part

of it resides solely with the subsystems or components of the system, while part of it

stems from the characteristics of the interactions among several of the

subsystems.[Asimow, 1962] While operational uncertainty resides at the system level,

readiness uncertainty is surely attributable to the component technologies. Fidelity

uncertainty then resides with the actual components as well as their interactions.

It is thus the designer’s responsibility to examine how the system will behave in

the future by virtue of its own inherent characteristics. The decision for a particular

design or class of designs will then have to be based on the level of satisfaction of a

certain set of desired outputs in a particular range of environments.[Asimow, 1962] This

design solution is typically called robust design solution, since its performance

parameters are invariant or almost invariant, i.e. robust, with respect to changes in the

production or operational environment.

Probability Theory in Systems Design

Simply acknowledging the existence of uncertainty in the decision making

process of systems engineering, however, does not indicate how to account for it in the

design process. “Decision making under risk occurs when the decision-maker does not

suppress acknowledged ignorance about the future but makes it explicit through the

assignment of probabilities. Such probabilities may be based on experimental evidence,


expert opinion, subjective judgment, or a combination of these.”[Blanchard, Fabrycky,

1998] Asimow provides an intuitive reason for the use of probability estimates by

writing: “before we actually choose a particular action, what do we believe is our

probability of successfully carrying it out? It is precisely at this point in our chain of

reasoning that we make a subjective estimate of our chances. We can elect to leave this

estimate implicit and submerged beneath our impressions, or we can try to draw it out in

an explicit statement as a quantitative expression of our level of confidence.”[Asimow,

1962]

While the introduction of probability estimates captures the uncertainty in

decision making, it is not until these estimates are translated into usable information to

the decision-maker that their full benefit can be felt. Traditionally, systems analysis

provides this information. Cleland and King define systems analysis as a process that

involves “[first] systematic examination and comparison of those alternative actions

which are related to the accomplishment of desired objectives, [second] comparison of

alternatives on the basis of the costs and the benefits associated with each alternative,

[and third] explicit consideration of risk.”[Cleland, King, 1983] While this definition is

global enough to include a wide variety of decision making problems and tools to answer

them, Silver and Silver believe that systems analysis consists predominantly of operations

research (OR).[Silver, Silver, 1976] The focus on OR within systems analysis allows to

prescribe a body of tools and methods to it that highlights its aim of capturing

uncertainties in decision making/design through mathematical techniques. Silver and

Silver emphasize such techniques as sampling, allocation of resources, including linear


programming and queuing theory, quality control, forecasting, including linear

regression, and simulations with mathematical modeling. Particularly the last group

enables a conversion of the decision making problem into mathematical terms that allow

for examining “the effect that a change in one or more elements in a system will have on

the rest of the system – without actually making the changes [in the real system].”[Silver,

Silver, 1976] Unfortunately, the systems engineering community has been slow in

adopting and applying these tools. It has been the structural and reliability engineers

rather that took to it more easily.[SAE, 1998] Perhaps the inherent uncertainty in

predicting a failure of a structural component paired with the severe consequences of that

failure introduced a need for new, non-traditional techniques of evaluating a successful

design. It is not before long, until this need will be felt in systems engineering as well.

Finally, the use of probability estimates for values of uncontrollable parameters,

technology performance and readiness estimates, as well as model fidelity estimates is

stipulated by the use of simulation and mathematical models in the design process.

Available OR methods need the prediction capabilities of these models. On the other

hand, probabilistic OR methods allow for a full exploration of all possible alternatives,

hence making maximum use of the analytical prediction capability of the models. For

example, a Monte-Carlo simulation will randomly select solutions from the design space

specified, generating numerous alternatives and estimating the chance of satisfying the

customer need, an impossible task without the use of mathematical models. But the

probabilistic analysis also yields information about the range of possible outcomes for the

specified inputs, while a simple deterministic analysis offers nothing but single point


estimates, without knowing their likelihood of occurrence. If the generation of

alternatives for a Monte-Carlo simulation takes too much time due to the complexity of

the analysis or system, fast probability estimation techniques can be employed to yield

the same estimates based on a smaller number of design alternatives. Of course, this

comes at the price of accuracy of the probability estimates. For very simple

mathematical models, analytical probability estimates are possible that do not require any

execution of the models at all. Their probability estimate is exact, but the model itself,

particularly in systems engineering, is probably not very accurate.

Despite the outlined advantages, probabilistic design methodologies have been

faced with some outspoken skepticism, if not criticism, in the engineering community. A

good summary of perceived and actual limitations of modern probabilistic design

techniques has been collected and published in the SAE Standard AIR5086.[SEA, 1998]

Based on a survey among structural and reliability engineers, the G-11 Probabilistic

Methods Committee found two groups of limitations: perceived ones and actual ones.

Among the perceived limitations the committee found a suspected radical departure from

existing practices and lack of compatibility with existing tools, difficulty in use and result

interpretation, too much effort, time, and data necessary, and finally a lack of verification.

The perceived radical departure and lack of compatibility of the probabilistic

techniques with the existing tools is mainly due to misinformation. As a matter of fact,

most probabilistic techniques encompass and build on existing tools, rather than replacing

them. The SAE Standard explicitly states that “probabilistic methods are, in fact, usually

close descendants of deterministic methods, born from a need to quantify the effects of


input variability.”[SAE, 1998] Chapter III of this thesis provides an overview of these

probabilistic design techniques that are specifically build around existing engineering

tools. On the other hand, the perceived difficulty in use and result interpretation is

simply based on a lack of education in probability theory among engineers. For this

reason, the SAE Standard makes a strong call for an adaptation and inclusion of

probability theory into today’s college curricula.[SAE, 1998]

The perceived limitation of probabilistic design techniques due to the extensive

time and effort needed for execution is only real for certain techniques, and can in most

cases be alleviated through parallel processing or metamodels,[SAE, 1998] which is

discussed in more detail in Chapter III. Finally, the perceived limitation of a lack of

verification of the new techniques is an accusation that holds only true when looked upon

with deterministic eyes. But probabilistic techniques do not try to predict a single

number, but rather indicate a likelihood for a range of system characteristic values. The

aim is different and comparison therefore flawed. By the same token, the deterministic

tools used in systems engineering almost never predict the correct number achieved by

the product, but this discrepancy is accepted and attributed to the fact the prediction was

done with a “model” and not the actual system. The aim of probabilistic techniques is to

capture this “prediction error” and make it explicit rather than have it appear as the big

surprise during operation.[SAE, 1998]

Among the actual limitations of probabilistic techniques, the SAE Standard

further found a lack of guidelines, the aforementioned computational effort, and difficulty

in validation. The lack of guidelines particularly is due to the fact that probabilistic


techniques have only been emerging recently and have not completely made their way

into the design and certification process for engineering systems. This limitation is

therefore just temporal and will hence diminish with time, since probabilistic techniques

will very soon take on a larger role in design. Similarly, the concern about the

computational efforts needed by many probabilistic techniques is a major field of

research and may therefore be considered a temporal limitation. It is important that

probabilistic techniques are not being dismissed in their entirety because of this

limitation, since it is only relevant for designs involving system analyses that require a lot

of computational resources and time. Finally, the most serious limitation of probabilistic

techniques is their lack of validation. In most cases, it is unreasonable to produce and

operate a statistically significant number of systems just in order to verify their design

process. The SAE Standard recommends instead verifying methods through continuos

collection of data at the component rather than the system level. A further possibility is

the use of Bayesian probability rather than pure statistics, with the benefit of being able to

update probabilistic assumptions as new data becomes available.[SAE, 1998]

Research Quest

In summary then, what is the problem that needs solving in systems engineering

design and is addressed in this thesis? The process of systems engineering, from the

needs analysis to the decision making process, has been outlined in this chapter. But how

is the inherent uncertainty in the design process accounted for in the decision making

procedure? This question is depicted symbolically in Figure 1.1.


Customer

Environment

Need for Product

Requirements/Desirements

Decision Making

Criteria

Systems Engineering

Probabilistic Techniques

Uncertainty

PRODUCT?

Figure 1.1: Design As a Decision Making Process

Starting at the top, the customer supplies the need for a product. Entering the

systems engineering process, this need is translated through a needs analysis into a set of

requirements and desirements1 that have the potential of being relaxed and traded-off

against each other. Requirements and desirements supply a set of multiple criteria based

upon which a particular product (concept) decision is made. At the opposite end of the

systems engineering process, the environment introduces uncertainty to the process. This

uncertainty is caused by technology developments and the assumptions made about the

product’s operation and can be modeled via probabilistic and systems analysis

techniques. The question mark in the middle of the figure represents the missing link

between the multi-criteria decision making process and probabilistic techniques, which

ultimately determines the product the customer will use in its operational environment.

1 Desirements - Customer supplied wants that impact the product design.


Unfortunately, no existing technique has succeeded in combining multi-criteria decision

making with probabilistic design. This void is intended to be filled by the Joint

Probabilistic Decision Making technique researched and developed in this thesis.

To facilitate this research and to structure the development of the technique, the

following research questions are posed, summarizing the issues raised in this

introduction, and will be addressed throughout the thesis in varying detail:

• Does uncertainty in the systems engineering design obstruct the decision process?

• Can the use of probabilistic design be beneficial in the decision making process?

• Is there a numerical value representing customer satisfaction?

• Does a technique already exist that can help the decision-maker find a best solution

based on multiple criteria arising from a probabilistic design technique?

• Is it possible to create such a technique and what should it look like?

• Can this technique be used for optimization?

• Can this technique be used for product selection?

The technique proposed in this thesis has its theoretical foundation in two areas:

Multi-Criteria Decision Making and Probabilistic Design. Multi-Criteria Decision

Making techniques have been used for more than 30 years in decision making problems

that involve multiple, conflicting criteria.[Hwang, 1979] A summary of the most

important techniques is provided in Chapter II. Probabilistic Design, on the other hand,

is a fairly new discipline, with only a small body of established literature. Its most

common techniques are outlined in Chapter III. The mathematical background of the


proposed Joint Probabilistic Decision Making technique rests on the joint probability

theory, which is summarized in Chapter IV. Chapter V finally introduces the technique

itself, while Chapter VI demonstrates the technique’s application in aerospace systems

design, comparing it to current methods of probabilistic design. Chapter VII summarizes

the technique and its achievements by revisiting the questions asked in this introduction

and highlighting the major findings of this thesis.

Oliver Bandte Chapter II 24

CHAPTER II

DETERMINISTIC MULTI-CRITERIA DECISION MAKING

TECHNIQUES

“Decision making is characterized by its involvement with information, value

assessments, and optimization. Thus, whereas inventiveness seeks many possible answers

and analysis seeks one actual answer, decision making seeks to choose the one best

answer.”[Dixon, 1966] But the ‘one best answer’ can be difficult to obtain, particularly

when the decision is based on several objectives. Ching-Lai Hwang has been on the

forefront of the development of new techniques and the enhancement of existing ones

that aid the decision-maker. His two references [Hwang, 1981] and [Hwang, 1979] list a

multitude of techniques, grouped in two classes; the Multi-Attribute Decision Making

(MADM) and Multi-Objective Decision Making (MODM) techniques. This chapter

highlights the most important techniques in both references.

According to Hwang, “multiple attribute decision [making] problems involve the

selection of the ‘best’ alternative from a pool of preselected alternatives described in

terms of their attributes.”[Hwang, 1979] Attributes are generally defined as

characteristics that describe in part the state of a product or system, while objectives are

attributes with a goal and a “direction ‘to do better’ as perceived by the decision-

maker.”[Hwang, 1979] Specifically, “goals are things desired by the decision-maker

expressed in terms of a specific state in space and time. Thus, while objectives give the


desired direction, goals give a desired (or target) level to achieve.”[Hwang, 1979] In

many cases, however, the terms ‘objective’ and ‘goal’ are used interchangeably. With

this definition for objectives in mind, multi-objective decision making problems “involve

the design of alternatives which optimize or ‘best satisfy’ the objectives of the decision-

maker.”[Hwang, 1979] In other words, multi-attribute decision making problems are

product selection problems, multi-objective decision making problems are optimization

problems. Together all techniques for solving both problems can be classified as Multi-

Criteria Decision Making (MCDM) techniques. While criteria typically describe the

standards of judgment or rules to test acceptability, here they simply indicate attributes

and/or objectives. In general, a MCDM problem is described by

)(, ),( ),(max 21 xxxx

Nfff K (2.1)

subject to a set of constraints

gr(x) r = 1, …, L, (2.2)

where fi, i = 1, …, N, are the criteria and x is the K-dimensional vector of design

variables the criteria and constraints depend on. Further, a feasible set X = x|g(x) 0

is the design variables x which satisfy the constraint vector g(x) 0. But for each point

in X there is an associated vector f(x), so that X can be mapped into a set

S = f(x)|x ∈ X in the criteria space. Further, x* is called an optimal solution iff

x* ∈ X and f(x*) ≥ f(x) for all x ∈ X. x* is called a nondominated solution iff no x ∈ X

exists such that fi(x) ≥ fi(x*) for all i and fj(x) > fj(x

*) for at least one j ≠ i, i, j = 1, 2, …, N.

In other words, a nondominated solution, also called a Pareto-optimal solution, is


achieved when no criteria can be improved without simultaneous detriment to at least one

other criterion.[Sen, Yang, 1998]

Product Selection

“The three elements that concern us in critical decision making, as it appears in

the process, are the alternatives, benefits, and difficulties of implementation.”[Asimow,

1962] The techniques in this section are concerned with exactly these three elements of

decision making. This class of Multi-Attribute Decision Making (MADM) techniques

determines the best of a finite and often small set of alternative solutions, based on the

attribute levels achieved and their indicated preferences. The final selection of the best

alternative is made with the help of inter- and intra-attribute comparisons, which may

involve implicit or explicit information.[Hwang, 1981]

Common to all MADM techniques is the concept of a decision matrix, or goal

achievement matrix D. D is an M-by-N-matrix with elements xji that indicate the value of

the attribute Xi with respect to the alternative Aj.

=

MNMiMM

jNjijj

Ni

Ni

Ni

M

j

xxxx

xxxx

xxxx

xxxxXXXX

A

A

A

A

D

LL

MOMMM

LL

MMOMM

LL

LL

LL

M

M

21

21

222221

111211

21

2

1

(2.3)

Another concept typical for MADM techniques is the ideal solution A*. It is a

hypothetical solution to the decision problem, combining the best achievements of all


criteria to one solution, i.e. A* = (f1*, f2

*, …, fi*, …, fN

*). Note, that in MADM problems

the ideal solution is subjective, driven by the existing solutions. In MODM problems, on

the other hand, the objective ideal is the best solution that any alternative could possibly

obtain. Further, for MADM problems, A* is infeasible most of the time, meaning it is not

part of the set of alternatives. If it was, there would be no conflict among the criteria and

A* would be the solution to the decision problem.

In general, MADM techniques can be separated into noncompensatory and

compensatory techniques by their treatment of attribute information. The

noncompensatory techniques do not allow for a trade-off between criteria, i.e. one

unfavorable criterion value cannot be offset by reducing a favorable value of another

criterion. Hence, comparisons are made on a criterion by criterion basis. The

compensatory techniques, on the other hand, permit trade-offs between criteria, assigning

a number to each multidimensional representation of an alternative. Based on the method

of calculating this number, these techniques can further be classified as scoring method,

compromising method, and concordance method.[Hwang, 1981] The scoring method

selects the alternative with the highest score, reducing the decision problem to obtaining

the appropriate multi attribute utility function. The compromising method identifies the

alternative closest to the ideal solution. The concordance method arranges a set of

preference rankings which best satisfies a given concordance measure. A selection

process to arrive at the right MADM technique for a given product selection problem is

outlined in the flowchart of Figure 2.1, as suggested by [Sen, Yang, 1998]. The

highlighted techniques are explained in more detail.


Which decision ruleis appreciated ?

Ordinal rankingof all attributes

Preferentially independentattribute set and

linear utility function

Relative closenessto ideal and

negative ideal points

Concordance anddiscordance

dominance indices

Simple AdditiveWeighting Method

LexicographicMethod

TOPSISMethod

ELECTREMethod

Decision tableRanking of alternatives

by attributes

Linear AssignmentMethod

Pairwise comparisons of all alternatives

Linear AssignmentMethod

What type of inputdata is available ?


and attributes

AHPMethod


LIMAPMethod

What type of inputdata is available ?

Given Generated

Minimal attribute value acceptable for each current attribute

ConjunctiveMethod

Greatest value of an attribute for an alternative

Disjunctive Method

What cut-off valuesare favorable ?

Is preference information required ?

Overall utilityfunction

UTAMethod

Local utilityfunction

ILUTAMethod

Implicit utilityfunction

EDMCMMethod

Is weight given or will it be generated ?

What types of utilityfunctions are appreciated ?

Non-dominance

DominanceMethod

Maximin

MaximinMethod

Maximax

MaximaxMethod

What decision rulesare appreciated ?

No Yes

Utility functions Relative weightStandard level

on each attribute

How is preferenceinformation represented ?

Given Generated

Figure 2.1: Decision Tree for MADM Technique Selection (based on [Sen, Yang, 1998])


Techniques for No Preference Information Given

MaxiMin Technique

MaxiMin is a widely used technique for MADM problems, particularly in

economics and game theory, and belongs to the class of noncompensatory techniques.

Any decision making situation where the analogy “a chain is only as strong as its weakest

link” applies can be solved with this technique. It identifies the weakest criterion for

each alternative and selects the alternative A+ that has the most acceptable level in its

weakest criterion. In other words, it is selecting the maximum (across alternatives) of the

minimum (across criteria) values, hence MaxiMin. Explicitly:

))(min(max| jiij

j xAA =+ , j = 1, 2, …, M, i = 1, 2, …, N. (2.4)

Principally, this technique reduces the multi-criteria problem to a single criterion

decision, since the single weakest criterion represents the whole alternative, ignoring all

other criteria. Thus, it only utilizes a small amount of the available information during

the decision making process. This obvious shortcoming is illustrated by the fact that an

alternative clearly superior in all other criteria will be thrown out, if its weakest criterion

performs worse than the weakest criterion of an alternative achieving average values for

all other criteria. However, the MaxiMin technique can be applied whenever the

decision-maker has a particularly pessimistic outlook and the criteria are truly of equal

importance and a failure in any criterion would prevent the alternative (system) from

performing in the desired manner.[Hwang, 1981]


Techniques for Standard Level of Attributes Given

Additive Utility Function Method (UTA)

The Additive Utility Function Method (UTA) attempts to rank alternatives based

on utility function values. First, a subset of alternatives is selected and ranked

subjectively. This subset of alternatives is further used to determine a utility function for

all alternatives based on a monotonic piecewise linear utility function for the attributes

and their subjective preferences. Finally, all alternatives are ranked by their utility.[Sen,

Yang, 1998] While the UTA method provides an alternative means to represent

preferences among alternatives, it has some disadvantages that seem to be problematic.

First, the method assumes an additive utility function for the attributes, which in turn

requires the attributes to be independent.[Sen, Yang, 1998] Second the method assumes

the attribute to be piecewise continuos, which is difficult to conclude from a finite set of

attribute values. Third, the utility function for all alternatives is based only on a subset

and their preference information is subjective. Last but not least, UTA assumes the

utility function to be (piecewise) linear and monotonically increasing or decreasing and

thereby limiting the range of problems the utility function can model.

Techniques for Standard Level of Attributes Given

Conjunctive Technique

The Conjunctive Technique is one of the oldest MADM tools and belongs to the

class of noncompensatory techniques. Minimum attribute levels are provided which each

criterion has to satisfy. Alternatives that do not satisfy all criteria levels are rejected.


This process eliminates a certain number of alternatives, depending on how aggressive

these goals are set. In order to avoid dismissing all alternatives on the first trial, an

iterative approach is typically selected that eliminates one alternative per iteration. Even

though no relative importance information is needed for this technique, alternatives are

not being credited for especially good attribute values. The concept of a minimum level

that needs to be satisfied for each attribute has a very intuitive appeal, however, and has

therefore been used in other MCDM techniques.[Hwang, 1981]

Techniques for Cardinal Preference of Attributes Generated

Analytic Hierarchy Process Method

The Analytic Hierarchy Process (AHP) method was originally introduced by

Saaty and is intended to solve such product selection problems that have a hierarchical

structure of attributes.[Saaty, 1980] Attributes in one level are compared in terms of

relative importance with respect to an element in the immediate higher level, treating the

pairwise comparison with the eigenvector method as outlined in [Sen, Yang, 1998]. This

process is executed from the top down starting with the overall goal as the single top

element of the hierarchy and closing with the alternatives at the very bottom, ranking the

attributes/alternatives at each level with respect to the overall goal.

While AHP method is well known, it has several disadvantages as outlined in

[Sen, Yang, 1998]. First, it requires attributes to be independent with respect to their

preferences, which is rarely the case in product selection cases. Second, all attributes and

alternatives are compared with each other (at a given level), which may cause a logical

conflict of the kind: A > B and B > C but C > A. The likelihood of such conflicts


occurring in the hierarchy tree increases dramatically with the number of alternatives and

attributes. Last but not least, AHP has the potential of introducing a rank reversal of

alternatives, depending on the number of alternatives assessed, which is particularly

troublesome for normative decision making environments.[Sen, Yang, 1998]

Techniques for Ordinal Preference of Attributes Given

Lexicographic Technique

In some decision problems a single criterion is predominant. If one alternative

yields a high value for this criterion, it is preferred over the other alternatives, which

yield lower values. However, if some alternatives have equal values for the most

important criterion, the next most important criterion might determine the best

alternative. If alternatives are tied again, the next most important criterion might yield an

answer, and so on. Thus, ordinal ranking of the criteria can provide a down selection

process of alternatives. A special form is the lexicographic semiorder technique, which

considers the next important criterion not only when the criterion values are equal but

also when their differences are negligible, keeping more alternatives in the decision

making process. Because of its need for only limited information, the lexicographic

technique has received serious consideration in the past.[Hwang, 1981]

Techniques for Cardinal Preference of Attributes Given

Technique for Order Preference by Similarity to Ideal Solution (TOPSIS)

TOPSIS is one of the compromising methods among the compensatory

techniques, utilizing preference information provided in the form of weights wi for each


criterion. It is based upon the concept that the best alternative should have the shortest

distance to the ideal solution and be farthest away from the negative-ideal solution. The

ideal solution A* is composed of the best normalized criterion values ∑ == M

j jii xxri 1

2**

obtained by all existing solutions, while the negative-ideal A- is composed of the worst

normalized criterion values ∑ =−− = M

j jii xxri 1

2 obtained. One approach uses the weighted

minimum Euclidean distance Sj* to determine the closest solution to the ideal. However,

depending on the location of the alternative Aj in the solution set S, the shortest Euclidean

distance to the ideal may not be the longest distance from the negative-ideal. Consult

Figure 2.2 for an example where A1 has a shorter distance to the ideal and the negative-

ideal solution than A2. TOPSIS considers both distances simultaneously by computing

the relative closeness Cj, j = 1, 2, …, M, to the ideal solution:

−

−

−=

jj

jj SS

SC

*

with ( )∑=

−=N

iiijiij rwrwS

1

2** and ( )∑

=

−− −=

N

iiijiij rwrwS

1

2. (2.5)

A2

Aj

A1

A*

A-

Attribute X1 (increasing preference)Att

ribu

te X

2 (i

ncre

asin

g pr

efer

ence

)

S

Figure 2.2: Euclidean Distance to the Ideal and Negative-Ideal Solutions [Hwang, 1981]


The solution with the highest relative closeness is the best solution among the

alternatives. The advantages of this technique are clearly its simplicity and the

indisputable ranking order obtained. However, the dependency on cardinal preference

information, such as weights, yields solutions highly dependent on their values. TOPSIS

further requires the criteria to have a monotonically increasing or decreasing utility to the

decision-maker, a condition that is violated in all situations where a particular attribute

value is supposed to be achieved, e.g. in tolerance design.[Hwang, 1981]

Optimization

Generally, optimization is defined as the process of maximizing a desired and/or

minimizing a detrimental quantity or outcome.[Dieter, 1991] On the other hand,

optimization is just a subset of decision making as Hazelrigg points out: “Decision

making is the taking of choices from sets of options in order to obtain the most desired

outcomes. This is precisely the process of optimization.”[Hazelrigg, 1996] As a decision

making problem, optimization has to be based on criteria that are to be maximized or

minimized. In systems design these criteria are typically the attributes of the system,

indicating its performance, cost, or reliability. Hence, optimization in systems design

requires the maximization or minimization of a multitude of criteria concurrently, which

in many cases results in a conflict. As a matter of fact, it is not intuitive what ‘optimal

solution’ means in a multi-criteria optimization problem. To solve this problem, several

multi-criteria decision making techniques for optimization have been introduced (see

[Hwang, 1979]), some of which are listed here with a brief explanation. A selection


process to arrive at the right MODM technique for a given optimization problem is

outlined in the flowchart of Figure 2.3, as suggested by [Sen, Yang, 1998]. The

highlighted techniques are explained in more detail.

Generate extremeefficient solutions

for MOLP problems

MOLPMethod

Represent efficientsolution set as someparametric functions

Envelope Method

Use weights asparameters to generate

efficient solutions

Parametric Methodor Weighting Method

Vary minimum allowable levels for objectives

to get solution set

Epsilon-constraintMethod

Generate approximateset of efficient solutions

Efficient SolutionGeneration Method

Which decision ruleis favorable ?

A posterioriarticulation

A prioriarticulation

Progressivearticulation

How is preferenceinformation elicited ?

Optimization of implicitconcave function based onset of positive multipliers

Zionts-WalleniusMethod

Optimization of localadditive utility functionbased on direct trade-off

REISTM Method

Optimization of implicitutility function based on

marginal rates of substitution

Geoffrion’sMethod

Generation of preferredsolution based on trade-offrates and surrogate function

Interactive SurrogateWorth Trade-Off Method

Optimization of implicitadditive separable

utility function

InteractiveGoal Programming


Optimization of utility function

Utility FunctionMethod

Closeness to ideal point

Ideal PointMethod

Satisfaction ofgoal values

GoalProgramming


Explicit trade-off

Directive trade-offs amongobjective functionsaround ideal point

ISTMMethod

Closeness to ideal point with satisfaction of one

objective at each interaction

STEMMethod

Generation and reductionof compromise sets bydisplacing ideal points

Displaced IdealPoint Method

Optimization of surrogateobjective function based ongoals and DM’s aspiration

SEMOPSMethod

Which decision rule or interactiveprocedure is favorable ?

Implicit trade-off

What type of preferenceinformation is appreciated ?

Ordinal

LexicographicMethod

GoalProgramming

Cardinal

What type of preferenceinformation is appreciated ?

Figure 2.3: Decision Tree for MODM Technique Selection (based on [Sen, Yang, 1998])


Techniques for No Articulation of Preference Information Given

Techniques in this category do not require any information about preferences

among the defined criteria. While this may seem as an advantage at first, since no

additional information is required from the customer or designer, it is probably more of a

disadvantage, since the decision will be made in isolation, not representing the true

preferences of the customer or company.

Technique of Global Criterion

The vector x* that minimizes some global criterion is called the optimal vector.

For all fi to be maximized the global criterion can be described by:

∑=

∗

∗

−N

i

p

i

ii

f

ff

1 )()()(

minx

xxx

. (2.6)

The optimal solution will differ significantly with a change in criteria chosen.

The difficulty with this approach is finding the value for p that will maximize the

satisfaction of the decision-maker. When p = 1 and fi(x) are linear functions, the decision

problem becomes a linear programming problem that can be solved with the Simplex

method for example. When p = 2 and fi(x) are linear functions, the problem becomes a

quadratic programming problem, solved by a gradient based technique for example.

When the fi(x) are nonlinear functions, the decision problem becomes a nonlinear

programming problem which can be solved by the Complex method.[Hwang, 1979]


MaxiMin Technique

The MaxiMin technique2 is the simplest form of multi-criteria optimization. The

basic principle for this technique is equivalent to the product selection technique

MaxiMin: find the smallest criterion value and change the independent variables such

that this particular criterion value is maximized. Repeat previous step until no

improvement is found. Or expressed mathematically:

)))(,),(),(),((min(max 21 xxxxx

Nii

ffff KK . (2.7)

Principally, this technique reduces the multi-criteria to a single criterion decision

problem, since the single weakest criterion represents the whole alternative, ignoring all

other criteria. Thus, it only utilizes a small amount of the available information during

the optimization process. This obvious shortcoming is illustrated by the fact that an

alternative clearly superior in all other criteria will be thrown out, if its weakest criterion

performs worse than the weakest criterion of an alternative achieving average values for

all criteria. Furthermore, whenever two or more (normalized) criterion values are equal,

the technique suffers from a discontinuous derivative.[Vanderplaats, 1999]

Techniques for ‘A Priori’ Articulation of Cardinal Preference Information Given

Techniques in this category utilize preference information provided before the

decision making as part of the mathematical modeling of the decision process. They also

require some judgement about specific criterion preference levels or specific trade-offs.

2 Also called MinMax technique in minimization problems.[Vanderplaats, 1999]


Utility Function Technique

For the utility function method, the optimization problem is converted to

maxU(f1, f2, …, fN) = U(f), where the U(f) is the utility function. The technique is

based on the idea that the decision-maker must have some kind of utility associated with

the criteria, accounting for the preferences. However, for complex problems such as

systems design, determining U(f) is very difficult. The major advantage of this technique

lies in the fact that the utility function, if assessed correctly, will ensure finding the most

desirable solution. There are several ways the utility function can be formed. A common

approach assumes that the utility function is a mere summation of criteria, reducing the

decision problem to:

( )∑=

=N

iii fuU

1

)( max xx

. (2.8)

A special form of Equation 2.8, which has been widely used in (aerospace)

systems design and other MODM problems, uses weights wi to indicate the importance of

each objective. It is formulated as

( )∑=

⋅=N

iii fwU

1

max xx

(2.9)

and is also known as the Overall Evaluation Criterion (OEC). The advantage of this

technique clearly lies in its simplicity. The disadvantages, however, are: (i) there are

very few cases where the utility functions is a linear combination of criteria and (ii) the

weights, wi, depend upon the achievement level of fi(x) itself and the relative achievement

of fi(x) compared to the achievement levels of the other criteria.[Hwang, 1979]


Techniques for ‘A Priori’ Articulation of Ordinal and Cardinal Preference Information

Given

Goal Programming

Goal programming finds the optimal solution by minimizing the deviation from

goals, specified for each criterion. The decision problem is formulated for p ≥ 1 by:

( )

+∑

=

+−pN

i

p

ii dd

1

1

minx

, with ( ) ,iiii bddf =−+ +−x and ,00, =⋅∧≥ +−+−iiii dddd (2.10)

where bi, i = 1, 2, …, N, are the goals to be obtained by the criteria, and di- and di

+ are the

respective under- and overachievements of the ith goal. A very common form of goal

programming requires an ordinal ranking of the criteria also. In this case the formulation

is:

( ) ( ) ( ) +−+−+− dddddd , ..., ,, ,,min 2211 NN hPhPhP (2.11)

where hi(d-, d+), are linear functions of the deviational variables and are called

achievement functions. The Pi’s are preemptive weights, i.e. Pi >>> Pi+1.

The algorithm that solves Equation 2.11 minimizes h1(d-, d+) first, thus

determining h1*. Next, h2(d-, d+) is minimized, but subject to h1(d

-, d+) ≤ h1*. In other

words, a lower ranking achievement function can not be satisfied to the detriment of a

higher ranking achievement function. This process is repeated until the last achievement

function is minimized. If the criteria are linear functions of x, a modified Simplex

algorithm for linear programming can be used to solve moderate size problems. For

large, complex problems, like systems design, a basic Simplex algorithm has to be used


within iterations. If the criteria are nonlinear functions, any single objective nonlinear

optimization technique can be used iteratively to solve the problem.[Hwang, 1979]

Requiring only a ranking but no numerical weights is one of the advantages of the goal

programming technique. However, the ordinal ranking implies a trade-off assumption

that is very strong and potentially restrictive. The fact that higher ranking criteria may

not be detrimented while minimizing lower ranking ones limits the possible solutions and

makes them extremely dependent on the ranking order. This can be particularly bad for

decision making problems with no clear preference of one criterion over another.

A variation of Goal Programming is the Goal Attainment Method, introduced by

Gembicki,[Gembicki, 1974] and relies on a goal and ‘weight’ for each criterion. The

method is of relevance, since it is one of the two multi-criteria optimization methods

available in MATLAB®.[Branch, Grace, 1996] The optimization problem is formulated

as: λminx

(2.12)

subject to: iii bwf ≤− λ)(x , wi > 0, i = 1, 2, …, N,

and the constraints 0)( ≤xrg , r = 1, 2, …, L.

Depending on the reference, the weights should either be normalized such that

they add to one,[Hwang, 1979] or should be set to zero for the criteria that must be

realized and set equal to the goal when no preferences are provided for the different

criteria.[Fleming, 1986] The Goal Attainment Method shares the disadvantages of all

Goal Programming methods, namely the high dependency of the solution on the weights

and goals provided by the decision-maker, but has fewer variables than the formulation in

Equation 2.10, and is hence computationally less intensive.


Techniques for Progressive Articulation of Implicit Trade-Off Information Given

This class of interactive techniques utilizes preference information progressively

defined during the exploration of the criterion space. They typically involve iteration

processes that require a preference or trade-off input from the decision-maker or his

consent with the current achievement level. This information is assumed to be

unavailable ‘a priori’ due to the complexity of the problem, but rather at a local level for

a particular solution. Some techniques allow the reversal of the preference order; few

guarantee a final solution within a limited number of iterations. Among their advantages

is the lack of a priori information needed; only local preference information is needed.

The iterative nature of this technique also constitutes a learning process for the decision-

maker, and the assumptions made are less restrictive compared to the previous

techniques. Among their disadvantages are the dependency of the solution on the

accuracy of the local preference information provided, and the fact that many methods

can not guarantee a preferred solution within a finite number of iterations. These

methods also require more effort from the decision-maker than the previous

techniques.[Hwang, 1979]

The STEM Technique

The STEP-method (STEM) is a solution technique for multiple objective linear

programming (MOLP) problems. An MOLP problem can be formulated as:

∑∑ ∑== =

K

jjNj

K

j

K

jjjjj xcxcxc

11 121 ,...,,max (2.13)


subject to: i

K

jjij bxa ≤∑

=1

, i = 1, 2, …, N,

0≥jx , j = 1, 2, …, K.

The STEM technique is a three step process that allows the decision-maker to

learn to recognize good solutions as well as the relative importance of the objectives, by

alternating phases of computing with phases of decision making. The first step consists

of the construction of a pay-off table, which lists in rows the values zij that are taken on

by the criterion fi when criterion fj reaches its feasible maximum solution. Step 2 consists

of the calculation phase, determining the feasible solution to Equation 2.13 which is

‘nearest’, in a Minimax sense, to the ideal solution fj*:

λminx

(2.14)

subject to: ( ) iii ff πλ ⋅−≥ x* , i = 1, 2, …, N, and i

K

jjij bxa ≤∑

=1

plus constraints from the previous cycle,

0≥jx j = 1, 2, …, K and 0≥λ .

πi gives the relative importance of the distances to the optimum. Note, it is only

locally effective and does not have such overriding powers as the weights in the utility

technique. If fi in the ith column of the pay-off table varies little from the optimum value

fi* when varying x, πi is assigned a small value since Criterion i is not sensitive to a

variation in weighting values. The third step then compares the criteria vector f m of the

mth iteration, corresponding to solution xm, with the ideal solution f*. In order to be able

to improve on the unsatisfactory criteria, the satisfactory ones need to be relaxed enough


to enable an improvement for the unsatisfactory criteria in the next iteration. Thus, the

Criterion i is allowed to be relaxed by ∆fi and the constraints fi(x) ≥ fi(xm) - ∆fi and

fj(x) ≥ fj(xm), i ≠ j and j = 1, 2, …, N, are added to the linear programming problem.

Finally the weight πi is set to zero and the calculation for cycle m+1 begins.[Hwang,

1979]

Techniques for ‘A Posteriori’ Articulation of Preference Information Given

These techniques determine a series of nondominated solutions, of which the most

desirable is selected based on some previously unindicated or nonquantifiable criteria.

This means that the trade-off information is used after the nondominated solutions have

been found. Hence, these techniques do not require assumptions or information

regarding a utility function, which is an advantage. However, in order to be accurate, a

large number of solutions is required, which makes it almost impossible to determine the

most desirable. Consequently, these techniques have rarely been used by themselves, but

rather in conjunction with other interactive techniques.[Hwang, 1979]

Parametric Method (Weighting Method)

Suppose that the relative importance of N objectives is known and represented by

a set of constant weights wi, i = 1, 2, …, N, with the sum of all wi equal to one. The

feasible, nondominated preferred solution can be obtained by solving:

( )

∑

=

N

iii fw

1

max xx

. (2.15)


Although, the assumption of linearity and additivity is difficult to satisfy, the

technique can be used to generate nondominated solutions by varying values for w.

Hence, the weighting coefficients are only parameters for finding the nondominated

solution points and do not reflect the relative importance of the criteria. A geometric

interpretation of the Parametric Method finds the points where the hyperplane

L = f(x)|wTf(x) = c, c is a constant, is tangential to the set of nondominated solutions.

The preferred solution is the one furthest away from the origin. This has been illustrated

for two criteria with a convex and nonconvex set in Figures 2.4 and 2.5 respectively. L is

the line with the slope –w1/w2. In Figure 2.4, the thick line indicates the set of

nondominated solutions, of which point A is the preferred solution. For the nonconvex

set, some solutions can not be found, since the technique will try to find only extremal

solutions. Thus the Parametric method will find Points B and C, but not A in Figure 2.5.

f1

f2

A

Slope = -w1/w2

S L

Figure 2.4: Parametric Method of Two

Criteria with a Convex Set

[Hwang,1979]

f1

f2

B

CA

Slope = -w1/w2

S L

Figure 2.5: Parametric Method of Two

Criteria with a Nonconvex Set

[Hwang, 1979]


Illustrative Example

In order to illustrate some of the previously listed optimization techniques, a

simple multi-criteria example is employed to compare their results. Consider the

following equations:

212

222

111 )(5.0)(5.05 yyyxyxf +−−−−= (2.16)

2221

212 coscos

2yxxyyf +++−= (2.17)

subject to: 22 1 ≤≤− x and 22 2 ≤≤− x . (2.18)

The optimal point (maximum) for f1 is clearly located at (y1, y2), while the optimum for f2

is (0, 0). Therefore, assuming y1 = 0.5 and y2 = 1.5, the maximum values for f1and f2 are

f1max = 5.75 and f2max = 5. On the other hand, the value for f1 at the optimum for f2 is

f1(0,0) = 4.5, and the value for f2 at the optimum for f1 is f2(0.5,1.5) = 3.7953, both being

quite different from their respective maximum value. To display these differences, both

functions were plotted in Figure 2.6 over the entire range for x1 and x2 at y1 = 0.5 and

y2 = 1.5.

Figure 2.6: Surface Plot for f1and f2

f2

f1

x2

x1


So the question arises, which is the best point when both functions need to be

maximized. To answer this question, three techniques are selected as a representation for

all the multi-criteria optimization techniques: MaxiMin, Utility Function, and Goal

Attainment Method. The Utility Function Method in itself does not include an

optimization algorithm, just the composed overall evaluation criterion (OEC), and is

therefore used in conjunction with the univariate sequential programming technique and a

line search.[Reklaitis, 1983], [Vanderplaats, 1999] In order to be able to compare the

results, no preferences are identified for either function. The Utility Function Method is

assigned equal weights, while the Goal Attainment Method is assigned weights of the

same value as the goals; the goals are set high, since both functions are to be maximized.

MaxiMin: ))),(),,((min(max 212211 xxfxxffx

. (2.19)

Utility Function: ),(5.0),(5.0 212211 xxfxxfOEC ⋅+⋅= (2.20)

Goal Attainment: λminx

(2.21)

subject to: 1010),( 211 ≥+ λxxf and 1010),( 212 ≥+ λxxf

All techniques are easily implemented in MATLAB® and well documented in the

User’s Guide to the Optimization Toolbox.[Branch, Grace, 1996] All results are listed

and compared in Table 2.1.

Table 2.1: Comparison of Three Multi-Criteria Optimization Techniques

MaxiMin OEC Goal Attain

x1* 0.0520 0.1543 0.0513

x2* 0.3152 0.7898 0.3154

f1(x1*, x2

*) 4.9477 5.4381 4.9477

f2(x1*, x2

*) 4.9477 4.6773 4.9477


The differences in the results clearly show how difficult it is to solve the multi-

criteria decision making problem. While MaxiMin and the Goal Attainment Method both

yield roughly the same results, it is not clear whether they are better than the result from

the OEC. Both methods drive the function values to be equal thereby drifting far from

the optimum for f1.3 The OEC on the other hand seems to yield a point that balances both

function values better. Unfortunately, the OEC is not an ideal technique either. It will

drive the multi-criteria solution of an optimization problem with vastly different function

values towards the optimum of the function with the largest function values. As a brief

example consider:

4.0000,10),(),(),( 212122211121 ⋅+⋅=⋅+⋅= wwxxfwxxfwxxOEC . (2.22)

Perturbations of the independent design variables x1 and x2 won’t change the

function value “0.4” much with respect to the function value “10,000”, and the solution

will therefore be driven by the “10,000” more than the “0.4”.

It seems that the solution can be improved in all three cases by normalizing the

function values. Unfortunately, this procedure will make the solution dependent on the

normalization value, but can at least alleviate the aforementioned problems as

demonstrated in Table 2.2. Here, all function values were normalized by their respective

maximum values:

MaxiMin: ))5

),(,

75.5),(

(min(max 212211 xxfxxffx

. (2.23)

3 The difference f1max - f1

* = 0.8023 is much larger than the difference f2max – f2* = 0.0523.


Utility Function:5

),(5.0

75.5),(

5.0 212211 xxfxxfOEC += (2.24)

Goal Attainment: λminx

(2.25)

subject to: 101075.5

),( 211 ≥+ λxxfand 1010

5

),( 212 ≥+ λxxf

Table 2.2: Normalized Comparison of Three Multi-Criteria Optimization Techniques

MaxiMin OEC Goal Attain

x1 0.1402 0.1396 0.1422

x2 0.7579 0.7317 0.7579

f1*(x1, x2) 5.4099 5.3899 5.4099

f2*(x1, x2) 4.7043 4.7222 4.7043

Note that the Minimax and Goal Attainment method, again, yield roughly the

same results. This time, however, much closer to the result from the OEC, with much

more balanced function values. Finally, it is up to the decision-maker to decide which

point is ideal.

The following question arises as an afterthought: what happens when the values

for y1 and y2 are not ‘known’ deterministically, but instead are uncertain. How do these

techniques handle uncertain information? Unfortunately, they don’t. All current multi-

criteria decision making techniques require known, i.e. 100% certain information. An

approach to systems design with uncertain information is the topic of the following

chapter.

Oliver Bandte Chapter III 49

CHAPTER III

PROBABILISTIC DESIGN METHODS

One of the major obstacles in applying probabilistic methods to design is the

accommodation of the large variety of computer codes used in modern systems

engineering. It is impractical to modify all of them just to accommodate a probabilistic

problem formulation. Hence, a more generic methodology is proposed in which a

‘wrapper’ is used to link the analysis codes, ‘drive’ the programs, and collect the output

parameter values. Using this formulation, probability functions are assigned to those

input variables whose actual values are considered to be uncertain, indicating the

likelihood of occurrence for all values within a given set. These variables are also called

noise variables. Since the inputs to the analysis codes are probability distributions, the

output parameters have to be distributions as well. In most cases the output distribution

of interest is a cumulative distribution function (CDF) that describes the likelihood of

achieving values less (or more) than specified values of interest. To generate these

distributions, many probabilistic analyses, e.g. the Monte-Carlo simulation, require a

large number of samples generated by the analysis.[Kleijnen, 1974] While the use of

computer models allows for an easy perturbation of input values, an increase in

complexity of the modeled system increases the complexity of the code and hence the

run-time of the computer. Fox lists three methods that incorporate such complex

computer programs in a probabilistic systems design approach.[Fox, 1994]


Method #1, displayed in Figure 3.1, directly links a computationally intensive

(large number of repetitions) thus inefficient probabilistic method, like the Monte-Carlo

simulation, to the traditional systems design codes used in deterministic design

approaches. Although computer speed has significantly increased in recent years, this

method can only be used with simple, i.e. fast analysis tools and/or parallel processing

(distributing system analysis over several processors). The extreme complexity of some

current design codes yields computation times that may prohibit a large number of

program evaluations within the allotted time frame for the design process. Thus,

Method #1 may not be a feasible option for a probabilistic design procedure if parallel

processing cannot be applied.

Inefficient ProbabilisticMethod to Obtain CDF

FastAnalysis Tool

Exact

InputVariables

ObjectiveCDF

Figure 3.1: Probabilistic Design Method #1 [Fox, 1994]

Method #2, displayed in Figure 3.2, proposes the use of a metamodel, which

approximates the exact design codes. The advantage of creating such a metamodel is a

significantly reduced execution time, allowing a Monte-Carlo simulation to run on the

metamodel rather than on the actual computer code. Several different metamodels have

been proposed and applied. Some of the more common regression models are based on

experimental designs [Kleijnen, 1987], artificial neural networks [Cheng, Titterington,

1994], or Fuzzy Graph based metamodeling.[Huber, Berthold, Szczerbicka, 1996]


Inefficient ProbabilisticMethod to Obtain CDF

Metamodel

Exact

InputVariables

ObjectiveCDF

ComplexAnalysis Tool

Approximation


Method #3, displayed in Figure 3.3, takes a different approach, approximating the

probability distribution function rather than the design code. This approximation is based

on the notion that in order to obtain the cumulative distribution function (CDF) not all

probability levels need to be identified. The method selects several percentile levels and

calculates the corresponding objective value. Note that this calculation is based on the

exact computer code, not on an approximating metamodel. These objective values and

their probabilities can than be used to fit the typical S-shape of a CDF. Method #3 has

found the widest application in the area of structural reliability,[Khalessi, Lin, 1993],

[Wu, Burnside, Dominguez, 1987] while more current publications suggest a use for

systems design also.[Mavris, Bandte, 1998], [Mavris, Kirby, 1998], [Mavris, Kirby,

Qiu,1998]

Efficient ProbabilisticMethod to Obtain CDF

ComplexAnalysis Tool

Exact

InputVariables

ObjectiveCDF

Approximation



Metamodel/Monte-Carlo Simulation

The metamodel/Monte-Carlo simulation combination (Method #2) has found the

widest application in aerospace systems design. In particular, the use of statistical

regression models based on Taylor series expansions in combination with experimental

designs is very popular. Examples can be found in [Chen, et al., 1995], [DeLaurentis,

Mavris, Schrage, 1996], [DeLaurentis, Calise, Schrage, Mavris, 1996], [Fox, 1994],

[Giunta, et al., 1996], [Kaufman, et al., 1996], [Mavris, Bandte, Schrage, 1995], [Mavris,

Bandte, 1996], [Mavris, Bandte, Schrage, 1996], [Unal, Stanley, Joyner, 1994]. The two

main reasons for its popularity are its simple application to numerous computer

simulation problems, like aircraft synthesis, and the large number of statistical analysis

tools commercially available, such as JMP, MINITAB, SPSS, etc. Nonetheless, there

are two major problems in metamodeling of complex computer codes with a high number

of inputs. First, the number of input variables handled by this approach is limited.

However, this problem can often be solved through a screening process [Box, Draper,

1987] that identifies the major contributors to the variation in model output.

The second problem with Method #2 lies in the mathematical background for

such regression methods as Response Surface Methodology (RSM) and Design Of

Experiments (DOE), which are based on random rather than deterministic variables (see

[Box, Draper, 1987], [Box, Hunter, Hunter, 1978], [Kleijnen, 1974], [Kleijnen, 1987]).

Fundamental statistical knowledge is critical for obtaining reasonable approximations of

the computer model. Many of the statistical results offered by the commercial packages

are based on random error estimation and do not completely reflect the accuracy of the


metamodel, since random error does not exist in deterministic computer simulation. A

discussion on accuracy and behavior of statistical regression metamodels in computer

simulations can be found in [Kleijnen, 1987], [Sacks, Schiller, Welch, 1989], [Sacks,

Welch, Mitchell, Wynn, 1989], and [Welch et al., 1992]. In general, the best validation

of the accuracy of the metamodel is an extensive test at randomly distributed points over

the design space to compare predicted values with the exact computer simulation values.

Unfortunately, this test increases the computational effort put into the generation and

validation of the metamodel.

Despite the aforementioned problems, the Response Surface Methodology (RSM)

can, if applied correctly, provide some valuable insight into the systems design code

behavior. Hence, it has been used as a metamodel generator to facilitate probabilistic

aerospace systems design methods.[DeLaurentis, 1998], [Fox, 1994], [Mavris, Bandte,

1996], [Tai, 1998] RSM is based on a statistical approach to build and rapidly assess

empirical metamodels.[Box, Draper, 1987], [Box, Hunter, Hunter, 1978] By designing

and analyzing experiments or simulations, the methodology seeks to relate and identify

the relative contributions of various input variables to the system attributes. However,

modern aerospace systems are extremely complex, and most attributes of interest are a

function of many hundreds of design variables. The first step in constructing a Response

Surface Equation (RSE) as a metamodel is to conduct a screening test to identify the

variables, which have the greatest contribution to the system attribute variation. The

screening test is comprised of a two level fractional factorial Design of Experiments that

accounts for main effects of variables only (i.e. no interactions).[Box, Hunter, Hunter,


1978] It allows the rapid investigation of many variables to gain a first understanding of

the problem.

After identifying the variables that form the RSE, a Design of Experiments has to

be selected. A typical, efficient design is a face-centered central composite design,

displayed in Figure 3.4, a three level composite design formed by combining a two-level

full or fractional factorial with a star design.[Box, Draper, 1987]

Figure 3.4: Face Centered Central Composite Design

Typically, a second order model in k-variables is assumed to exist as an

approximation of the analysis code. This second order polynomial for a response R can

be written as:

∑∑∑∑=

−

===

+++=K

i

i

jjiij

K

iiii

K

iii xxbxbxbbR

2

1

11

2

10 (3.1)

where: bi are regression coefficients for linear termsbii are coefficients for pure quadratic termsbij are coefficients for cross-product termsxi, xj are the design variables of interest

Refer to [Box, Draper, 1987] and [Box, Hunter, Hunter, 1978] for a detailed

description of response surface generation. Both references provide several ways of

measuring and estimating the prediction accuracy of the RSE, but due to the lack of


randomness in the data (each experiment run with the computer is repeatable) their

validity is questionable.[Kleijnen, 1987], [Sacks, Schiller, Welch, 1989], [Sacks, Welch,

Mitchell, Wynn, 1989], [Welch et al., 1992] The best validation of an RSE is

undoubtedly a comparison of RSE and analysis code values at random points throughout

the design space.

The developed RSEs, representing the analysis codes in the design process, can

subsequently be used for a Monte-Carlo simulation to incorporate the effect of uncertain

variables in the design process. A Monte-Carlo simulation is effectively a random

number generator that selects values for each random variable with a frequency

proportional to the shape of the corresponding probability distribution. Usually,

dependent on the desired accuracy, 5,000 to 10,000 trials are needed for a good

representation of the response probability distribution.4 Without the aid of the RSE, this

task would be computationally excessive and in many cases impractical, considering that

a Monte-Carlo simulation would have to execute the design simulation code each time

(Method #1, Figure 3.1).

Fast Probability Integration

As an alternative to the sampling of a metamodel (Method #2), a fast probability

integration (FPI) technique can be used as an approach to Method #3. FPI techniques are

probability analysis techniques based on the Most Probable Point (MPP) analysis,

4 Refer to Appendix B for a detailed description of the number of necessary samples.


frequently used in structural reliability analysis. The MPP analysis utilizes a response

function Z(X) that depends on several random variables Xi (see Figure 3.5 for a 2-D

example). Each point in the design space spanned by the random variables has a specific

probability of occurrence according to their joint probability distribution function (see

Figure 3.6). However, each point in the design space also corresponds to one specific

response value Z(X). Hence, each response value has the same probability of occurrence

as the corresponding point in the design space.

x2Z(x) = z1

x1

Z(x) = z2

Z(x) = z3

Z(x) = z4

Figure 3.5: Objective FunctionContours

Figure 3.6: Joint Probability Distribution[Ang, Tang, 1984]

In systems analysis and other disciplines involving random variables, it is often

desired to find the probability of achieving response values below a critical value of

interest z0. This critical value can be used to form a limit-state function (LSF):

g(X) = Z(X) – z0 (3.2)

where values of g(X) > 0 are undesirable. The MPP analysis calculates the

cumulative probability of all points that yield g(X) ≤ 0 for the given z0 (see Figure 3.7).


Since the LSF ‘cuts off’ a section of the joint probability distribution (see Figure 3.8) a

point with maximal probability of occurrence can be identified on that LSF. This point is

called the Most Probable Point. It is found most conveniently in a transformed u-space

(see Figure 3.8), in which all random variables are normally distributed. Once the MPP

and the cumulative probability are identified, the process can be repeated for several z0

values, mapping the corresponding probability to z0. This cumulative probability

distribution for Z(X) can then be differentiated to obtain the probability density function

of the response.

MPP

g(x) = 0

x1

x2

g(x) < 0

g(x) > 0

Figure 3.7: Most Probable PointLocation

g(u)

u1

MPP

u2

Figure 3.8: Visualization of MPP[Southwest Research Institute, 1995]

The FPI techniques utilize several methods to find the MPP and the probability of

a given LSF value z0 for the response function. Some of these methods are very efficient

and eliminate the need for an expensive Monte-Carlo simulation. The advantage of FPI

techniques is the direct linkage of the probabilistic analysis to the analysis code,

eliminating the need for a metamodel and its limit in the number of variables. However,

all Fast Probability Integration methods approximate the LSF locally at the Most

Probable Point.


Advanced Mean Value Method

The Advanced Mean Value (AMV) method is one of the FPI techniques that

combines a simple Mean Value method with the MPP analysis and determines the CDF

for the response function Z(X). The Mean Value (MV) method is based on a simple

Taylor series expansion of the response function Z(X) (Equation 3.3), assuming Z(X) to

be smooth and the expansion to exist at the mean:

( ) )()()( )()()(11

XXXXX HZHXaaHXX

ZZZ MV

N

iiioii

N

i i

+=++=+−⋅

+= ∑∑

==µ

∂∂

. (3.3)

All derivatives are evaluated at the mean values for the design variables. ZMV(X)

represents the sum of the first order terms and H(X) represents higher order terms. For N

random variables, the ai’s can be estimated with N+1 function evaluations and a

numerical differentiation method. Based on this linear approximation the CDF for

ZMV(X) can be obtained directly, since the distributions for the random variables Xi are

fully defined and ZMV(X) is explicit. However, for nonlinear Z-functions the MV solution

for the CDF is not sufficiently accurate. One possibility for increasing accuracy is to

increase the order of the Taylor series expansion, which becomes difficult and inefficient

for implicit response functions and a large number of random variables.

A more efficient approach to increasing the accuracy is proposed by the AMV

method: ZAMV = ZMV + H(ZMV) (3.4)

H(ZMV) is defined as the difference between Z and ZMV at the Most Probable Point Locus

(MPPL) of ZMV, where the MPPL combines the MPP’s for several values of z0.[Wu,

Burnside, Dominguez, 1987] In other words, H(ZMV) in Equation 3.4 approximates H(X)


in Equation 3.3. ZAMV would be exact if the MPPL was known and exact, i.e.

MPPL(ZMV(X)) = MPPL(Z(X)). Since the MPPL is not known, the AMV method

approximates the locus based on ZMV, which is a good approximation for smooth

response functions.[Southwest Research Institute, 1995] Again, to avoid confusion with

Method #2, the AMV method does not approximate the response function to obtain the

CDF but rather the MPP (MV method). This approximation, however, is corrected by the

move in the AMV method, as depicted in Figure 3.9. The steps for a CDF generation

with the AMV method are also displayed in Figure 3.9.

Analysis Code:Response Function Z(X)

ZMV = a0 + a1X1 + a2X2 + …

CDF, MPP(X*)

Analysis Code:Response Function Z(X)

ZAMV = Z(X*)

1

0 zZAMVZMV

H

Figure 3.9: AMV Method [Southwest Research Institute, 1995]

One of the dominant advantages of the AMV method is the small number of

function calls necessary for each output parameter distribution. N+1 analysis code

executions are sufficient for the linear approximation of the response function ZMV and

ten additional program evaluations are needed to obtain the updated ZAMV for ten selected

levels of z0.[Southwest Research Institute, 1995] This translates into significant time


savings over the RSE/MCS method which usually requires several hundred function

evaluations for the generation of the RSE.[Box, Draper, 1987] Additionally, the AMV is

principally not limited to a small number of variables. The current limit of 100 variables

for the computer based implementations of the FPI technique is only due to vector

dimensioning in the source code and not the technique itself. Nonetheless, there is an

additional gain associated with the extended effort in the RSE generation. It can serve as

a valuable tool in gaining insight to the behavior of the underlying model. The AMV

method, on the other hand, will only return a probability distribution without providing

any further information about the analysis code. A more detailed comparison of

Method #2 and Method #3 can be found in [Mavris, Bandte, 1998].


To illustrate the three different probabilistic design methods, the example problem

from Chapter II is employed here again. For this example, the values for y1 and y2 are

assumed to be uncertain, and probability distributions are assign to them. It is easily seen

from Equations 2.16 and 2.17 that with changing y1 and y2 values the optimal point for f1

changes while the optimal point for f2 stays the same (0,0). This phenomenon is

exemplified in Figure 3.10 with 100 different values for y1 and y2.


-2 -1 0 1 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Figure 3.10: Change in Optima for f1 and f2 with Changing y1 and y2

In order to not overly complicate the problem, Normal-Distributions are assigned

to both random variables, Y1 ~ N(1,1) and Y2 ~ N(1,1). The optimization problem then

tries to seek the solution that maximizes the probability of achieving a certain goal, lets

say f1 > 5 and f2 > 5. Since f1 and f2 depend on the random variables Y1 and Y2, they

become random variables themselves and the probabilistic optimization problem can be

formulated as:

)5)(5.0)(5.05(max)5(max 212

222

11,

1, 2121

≥+−−−−=≥ YYYxYxPFPxxxx

(3.5)

)5coscos(max)5(max 2221

21

,2

, 22121

≥+++−=≥ YxxYYPFPxxxx

(3.6)

subject to: 22 1 ≤≤− x , 22 2 ≤≤− x , Y1 ~ N(1,1), and Y2 ~ N(1,1).

To demonstrate the differences between the three models outlined in this chapter,

cumulative distribution functions (CDF) generated by each method are compared with

each other. Note that a metamodel approximation of f1 in form of a second order

polynomial simply reproduces the original function, since it is a second order polynomial

x1

x2

Optimum for f2

Optima for f1


already. Consequently, the CDFs for f1 generated by Method #1 and Method #2 are

identical. Unfortunately, the tool used to generate the comparison, CRAX,[Robinson,

1999] does not compute CDFs based on probabilities of random variable values being

larger than a specified value. Hence, for the comparison, the problem is transformed to

)5()5( 11 −≤−=≥ FPFP , (3.7)

which is used in the subsequent plots.

First, as an example test case for x1 = x2 = 1, the CDFs for F1 generated by the

probabilistic design Methods #1 and #3 are compared with each other. As displayed in

the graph of Figure 3.11, the two CDFs are quite different. Generally, the CDF generated

by the sampling method is assumed to be the truth model,[Fox, 1994],[Fox, Reh, 2000],

[Mavris, Bandte, 1998] which infers that the CDF generated by the AMV method is false.

This misprediction is related to difficulties the AMV method5 has with such non-

monotonic functions as f1.[Fox, Reh, 2000], [Wu, Millwater, Cruse, 1990]

Second, the CDFs for F2 are compared via the graph in Figure 3.12, again for

x1 = x2 = 1. In this case, however, all three methods produce a distinctly different CDF,

since f2 is not a second order polynomial like f1. As a matter of fact, f2 is so complex with

its ‘y22cosx1’ term that a regular central composite design, in Figure 3.4, was not

sufficient to produce a reasonable approximation of f2. Instead, a 3-level full factorial

design was employed to generate the second order polynomial:

22

22

2112 0559.1354.04782.26087.7 yxxyf RSE +−−−= . (3.8)

5 The AMV and the AMV+ method both yielded the same CDF.


−11.00 −9.60 −8.20 −6.80 −5.40 −4.00 −2.60 −1.20 0.20 1.60 3.00 0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

1.000

Figure 3.11: CDF Comparison of Method #1 and Method #3 for F1

−20.0 −17.0 −14.0 −11.0 −8.0 −5.0 −2.0 1.0 4.0 7.0 10.0 0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

1.000

Figure 3.12: CDF Comparison of Method #1,#2, and #3 for F2

Nonetheless, the RSE’s failure to accurately represent f2 causes the CDF

generated with probabilistic design Method #2 to be significantly different from the ‘truth

model’ of Method #1. Obviously significant improvement could be achieved if the RSE

was altered to include higher order effects accounting for ‘y22cosx1’ term. But the

purpose of this illustrative example is not an accurate prediction of some CDF, but rather

the illustration of the three probabilistic design methods and the identification of some

potential pitfalls. On the same token, the CDF generated by the AMV method, while

Method #3(AMV) Method #1

(Monte-Carlo)

-f1

Pro

babi

lity

Method #3(AMV)

Method #1(Monte-Carlo)

-f2

Pro

babi

lity

Method #2(RSE/MCS)


closer to the truth model this time, shows an uncharacteristic dip at very high probability

levels. This dip defies one of the conditions of a CDF, monotonically increasing function

values, and is due to AMV’s aforementioned problems with non-monotonic limit state

functions.

Since the functions f1 and f2 are explicitly available and simple to evaluate,

Method #1 will be used from here on to solve the optimization problem in Equations 3.5

and 3.6. In essence, Equation 3.5 and Equation 3.6 are two new objective functions that

are dependent on the design variables x1 and x2. Consequently, the objective functions

span a surface over the design space, where each point in the design space corresponds to

a probabilistic analysis, e.g. a Monte-Carlo simulation, which determines the function

value. The surfaces for Equations 3.5 and 3.6 are displayed in Figures 3.13 and 3.14

respectively.

Figure 3.13: Response Surface forEquation 3.5

Figure 3.14: Response Surface forEquation 3.6

Equation 3.5 exhibits two maxima, the smaller one barely visible, located around

x1 = 0 and x2 = 0. Equation 3.6 exhibits only one local maximum, which is to be

expected, since the location of the maximum for f2 is not affected by changes in y1 and y2

Pro

babi

lity


values. A simplex search method is employed to find the optima for Equations 3.5 and

3.6. The best solution found is x1 = 0.8230 and x2 = 0.8759 for Equation 3.5 and

x1 = 0.0155 and x2 = 0.0049 for Equation 3.6 with function values of

5501.0)5)8759.0 ,8230.0(( 1 =≥FP and 2932.0)5)0049.0 ,0155.0(( 2 =≥FP , which is

equal to the true maximum at x1 = 0 and x2 = 0.

Note that the surfaces in Figures 3.13 and 3.14 are very smooth, which can be

attributed to the fact that each point on the surface is based on 10,000 Monte-Carlo

simulation samples. Reducing the number of samples increases the ruggedness of the

surface. Compare Figures 3.15 and 3.16, which display the same surface for

Equation 3.5 for 10,000 and 1,000 samples respectively. The rougher surface in

Figure 3.16 makes it more difficult for simple optimization routines to find the true

optimum, since they are likely to get stuck in one of the smaller local ones. On the other

hand, a very high number of samples guaranties a smooth surface, allowing for the use of

a simple and quick optimization routine.

Figure 3.15: Response Surface forEquation 3.5 (10,000 Samples)

Figure 3.16: Response Surface forEquation 3.5 (1,000 Samples)

Oliver Bandte Chapter IV 66

CHAPTER IV

MULTI-VARIATE PROBABILITY THEORY

Chapter I established that engineering system designs are evaluated based on a

multitude of criteria and the design process typically relies on uncertain assumptions. To

accommodate both aspects of design concurrently, an extension of the commonly used

univariate probability theory is needed. It is insufficient to look at each criterion and its

distribution independently, since all attribute values are generated by the same design

process representing a common system and are thus interdependent. The assumption of

independent criteria is therefore typically unfounded. The aforementioned necessary

extension is consequently a probability theory for jointly distributed random variables.

Definition: Let X1, X2, ….,XN be a set of random variables defined on a (discrete)

probability space Ω. The probability that the events X1 = x1, X2 = x2, ….,and XN = xN

happen concurrently, is denoted by f(x1, x2,…., xN) = P(X1 = x1, X2 = x2, …., XN = xN)6 for

the set of desired solutions A ⊆ Ω. If the function f(x1, x2,…., xN) is discrete, it is called

the joint probability mass function of X1, X2, ….,XN and has the following properties

[Hogg, Tanis, 1993]:

1),....,,(0 21 ≤≤ Nxxxf 1),....,,(),...,,(

21

21

=∑ ∑Ω∈Nxxx

Nxxxf (4.1)

6 P(X1 = x1, X2 = x2, …., Xn = xN) = P[(X1 = x1) ∩ (X2 = x2) ∩….∩ (XN = xN)].


[ ] Ω⊆=∈ ∑ ∑∈

AxxxfAXXXPNxxx A

NN ,),..,,(),..,,(),...,,(

2121

21

(4.2)

If f(x1, x2,…., xN) is continuous, it is called a joint probability density function of

X1, X2, …., XN and has the following properties [Hogg, Tanis, 1993]:

),....,,(0 21 Nxxxf≤ 1....),....,,( 2121 =⋅⋅⋅∫ ∫Ω

NN dxdxdxxxxf (4.3)

[ ] Ω⊆⋅⋅=∈ ∫ ∫ AdxdxdxxxxfAXXXPA

NNN ,..),..,,(),..,,( 212121 (4.4)

If the lower bound of A, the set of desired solutions, is equal to the infimum7 of Ω

for all Xi, i.e. if A = (infi(Ω), ai] ⊆ Ω, for all i = 1, 2,..., N, a function F(a1,a2,….,aN) can

be defined, such that:

[ ] ∑ ∑∈

⋅⋅⋅=∈=),...,,(

212121

21

),..,,(),..,,(),..,,(Nxxx A

NNN xxxfAXXXPaaaF (f is discrete) (4.5)

[ ] ∫ ∫⋅⋅=∈=A

NNNN dxdxdxxxxfAXXXPaaaF ..),..,,(),..,,(),..,,( 21212121 (f is continuous). (4.6)

F is called the joint cumulative probability distribution function.[Ross, 1993] For

Ω = ℜN and a continuous function f:8

[ ] ∫∫∞−∞−

⋅⋅=∈= −∞−∞−∞1

..),..,,()],..,,(),,..,,((),..,,(),..,,( 121212121

a

NN

a

NNN dxdxxxxfaaaXXXPaaaFN

. (4.7)

The univariate probability function fXi for each criterion Xi, obtained from the

traditional probabilistic design process, can also be generated with the joint probability

7 Greatest lower bound.8 ℜn denotes the set of all real valued N-touples. The common notation: F(a1, a2,.., aN) = P(X1 ≤ a1, X2 ≤ a2, …., XN ≤ aN) will be used subsequently also.


function f. fXi is called marginal probability mass or density function (PDF) of Xi and

defined by:

∑ ∑∈

⋅⋅⋅=),...,(

2

2

1),....,(

Nxx RNX xxff (f is discrete) (4.8)

∫ ∫⋅⋅⋅=R

NNX dxdxxxff ....),....,( 221(f is continuous). (4.9)

Figure 4.1: Joint and Marginal PDF of Continuous Criteria X and Y [Ang, Tang, 1984]

To further illustrate the concept of joint probability, an example for two

continuous criteria, X and Y, is displayed in Figure 4.1. The joint probability function,

fX,Y(x,y), creates the surface of a probability ‘hump’ in the x-y-f-space, characterized by

rings of constant probability density. The distribution curves over the x- and y-axis are

the aforementioned marginal probability functions fX(x) and fY(y), respectively. Also


displayed in Figure 4.1 are two ‘cuts’ through the probability ‘hump’, marking the

probability distributions fX,Y(x = a, y) and fX,Y(x, y = b) and their respective areas

underneath fX(a) and fY(b).

The last necessary concept to mention here for the development of a joint

probabilistic formulation is the concept of dependence of criteria. Two random variables

X and Y are said to be independent, if

)()(),(, yfxfyxf YXYX ⋅= ; (4.10)

otherwise X and Y are said to be dependent. This dependence is a mathematical notion

and should not be confused with ‘causal dependence’. A simple example for

mathematical dependence without causal dependence is the number of times a person

takes an umbrella to work and the number of times he wears long pants in a given month.

The two numbers increase similarly with the number of rainy days in that month, i.e. they

are (mathematically) dependent. They are, however, not causally dependent, since

wearing pants does not depend on taking an umbrella or vice versa, but rather on the rain

the person has to face on the way to work.

From here on, mathematical dependence will be referred to as correlation.

Correlation is measured by the covariance of two criteria, X and Y, defined by [Ang,

Tang, 1984]:

Cov(X,Y) = E[XY] – E[X]E[Y]. (4.11)

It is more convenient, however, to use a covariance normalized by the standard

deviations, σX and σY, for both criteria, called correlation coefficient [Ang, Tang, 1984]:


YX

YXCov

σσρ ),(= . (4.12)

The correlation coefficient is defined over the interval (-1,1), indicating strongly

positively correlated criteria at values close to 1 and strongly negatively correlated

criteria at values close to –1. The criteria are independent, if ρ = 0. In aerospace systems

design ρ can be quite difficult to calculate by Equation 4.12. It is much more effective to

view the correlation coefficient differently for calculation purposes. Jointly collected

data from a probabilistic or any other analysis can be thought of as vectors of numbers.

The correlation coefficient measures the orthogonality, i.e. independence, of both vectors,

a and b. ρ is simply the cosine of the angle between the two criterion vectors:

baba ⋅== θρ cos . (4.13)

It does not reflect any causal relationship, it merely indicates their alignment. For

ρ = 1, vectors are parallel and point in same direction, for ρ = -1, vectors are parallel and

point in opposite direction. For ρ = 0, vectors are orthogonal and the criteria are

independent. The correlation coefficient plays a significant role in the formulation of

joint probability distribution models as described in the next section.

Oliver Bandte Chapter V 71

CHAPTER V

A NEW PROBABILISTIC MULTI-CRITERIA DECISION

MAKING TECHNIQUE

Introduction

A key problem in complex systems design is measuring the ‘goodness’ of a

design, i.e. finding a criterion through which a particular design is determined ‘best’.

Traditional choices in aerospace systems design, such as gross take-off weight,

acquisition cost, and payload, individually fail to fully capture the life cycle

characteristics of the system. Thus, a common approach has been to combine all criteria

together into one equation termed the overall evaluation criterion, OEC. This equation is

often very simple in its mathematical structure due to lack of any better model for the

decision process. Recognizing this lack of proper decision process modeling, a different

approach is derived in this thesis, using the system attributes concurrently as decision

criteria for the evaluation of designs. This evaluation is not based on a summation of

criteria, like an OEC, but rather the probability of satisfying all criteria at the same time,

a notion similar to a Pareto-optimality.9 The main difference with respect to Pareto-

optimality lies in the optimizable objective function, called Probability of Success (in

9 State of economic affairs where no one can be made better off without simultaneously making anotherworse off.[Nas, 1996]


satisfying all criteria).

This multi-criteria approach to decision making lends itself more suitably to

aircraft design than a single-criterion approach, since customers typically like to see all

decision criteria satisfied. For example, a probabilistic multi-criteria approach can yield

the design solution, which maximizes the probability of low cost, high capacity, speed,

and dependability, while a single objective design will only yield an optimum in one of

these criteria, neglecting all others.

The proposed method also accommodates the modeling and use of uncertain

information intrinsic to system descriptions in conceptual and preliminary design. For

example, the designer may have a flight path or mission scenario for the aircraft, but is

unclear about the operating conditions. The modeling option used in this method treats

the incomplete information probabilistically, accounting for the fact that certain values

may be more prevalent, while the actual value during operation is unknown. By

assigning probability estimates to the values within the range of interest, the method

guarantees that all values are kept as possible solutions. Hence, these values and their

corresponding likelihood constitute a new type of assumption in the systems design

process, which yields the aircraft’s attributes, and thus the decision criteria, as random

variables.

If multiple, interdependent criteria are needed for decision making, a joint-

probabilistic formulation is needed to accurately estimate the probability, since the

marginal, or univariate, distribution for each criterion does not indicate the likelihood of

any other criterion value. In fact, most aircraft attributes are interdependent, since they


are evaluated by the same design process or analysis. For example, the probability of

cost being below a particular value depends on the value of payload, speed, and system

reliability. The proposed joint-probabilistic approach to multi-criteria aircraft design,

called the Joint Probabilistic Decision Making (JPDM) technique, will facilitate precisely

this estimate, closing the gap between traditional decision making tools and uncertainty

estimation techniques, as indicated in Figure 5.1.

Customer

Environment

Need for Product

Requirements/Desirements

Decision Making

Criteria

Systems Engineering

Probabilistic Techniques

Uncertainty

PRODUCTJPDM

Figure 5.1: Filling the Gap in the Design Process

The JPDM technique can be categorized as a Multi Attribute Decision Making

(MADM) as well as Multi Objective Decision Making (MODM) tool. As pictured in

Figure 5.2, MADM tools are characterized by a small number of alternatives to be

evaluated based on a large number of criteria, reflecting a selection problem. MODM

tools, on the other hand, are characterized by a small number of criteria used to determine

the best solution from amongst a large pool of alternatives. This formulation reassembles

a design (optimization) problem, optimizing the design variables based on a (small)


number of objective functions. The JPDM technique, however, is not limited to a small

number of alternatives or decision criteria.

Alternatives

Alt 1 Alt 2 Alt 3 Alt 4 Alt 5 ….. Alt N

Crit 1 Value Value Value Value Value Value





…..

Cri

teri

a

Crit M Value Value Value Value Value ValueJPDM

MODM

MADM

Figure 5.2: Applicability of MADM, MODM, and JPDM

Algorithms

To integrate the joint probabilistic formulation of Chapter IV into the design

process, algorithms need to be created that allow for an easy implementation into a

numerical framework. The necessary transition from the mathematical formulation in

Chapter IV to a probabilistic model that yields the information relevant for multi-variate

decision making is described in this section. For that the following rules are established

based on which algorithms are deemed suitable for implementation into systems design:

• Algorithms may not be limited in number of random variables, i.e. criteria.

• Algorithms need to be flexible with respect to the criterion distributions.

• Algorithms have to satisfy the conditions of a joint probability distribution.

• Algorithms cannot require numerical integration.

N

M


The last requirement stems from the fact that a numerical integration would

require too many function evaluations, particularly in problems with many criteria. For

example, the numerical determination of a joint distribution with ten variables, using 100

points for integration in each dimension, requires as many as 1020 function evaluations, a

task too daunting even for modern desktop computers. Most algorithms found in the

literature, however, violate at least one of these requirements, giving rise to the following

two.

Empirical Distribution Function:

The first algorithm that can be used for a joint probabilistic formulation is the

nonparametric Empirical Distribution Function (EDF), named after the empirically

collected data samples upon which the probability is estimated. Its univariate probability

mass function for a random variable X is defined for M samples as:

)()(1

1 xaIxfM

jjMX == ∑

=where

=

==otherwise 0

for 1)(

xaxaI j

j . (5.1)

aj are the criterion sample values derived from a sampling method such as the Monte-

Carlo simulation, while x is the criterion value of interest. The EDF’s cumulative

probability function is consequently defined as:

)()(1

1 xaIxFM

jjMX ≤= ∑

=where

≤

=≤otherwise 0

for 1)(

xaxaI j

j . (5.2)

Recognizing the joint probabilistic notation from Chapter IV, the univariate EDF

can easily be extended to more random variables. The joint probability mass function

(Equation 4.2) can thus be formulated as:


( )∑=

==M

jNjNjjMN xxxaaaIxxxf

12121

121 ),..,,(),..,,(),..,,( (5.3)

where ( ) =

==otherwise0

),..,,(),..,,(for 1),..,,(),..,,( 2121

2121NjNjj

NjNjj

xxxaaaxxxaaaI .

Similarly, the joint cumulative probability distribution function (Equation 4.5) can be

formulated as:

),..,,(),..,,( 2211

11

21 NjNj

M

jjMN xaxaxaIxxxF ≤≤≤= ∑

=(5.4)

where ( ) ≤≤≤

=≤≤≤otherwise0

),..,,(for 1,..,, 2211

2211NjNjj

NjNjj

xaxaxaxaxaxaI .

The joint EDF satisfies all previously stated requirements, since it is not limited in

the number of criteria, depends on joint samples for the criteria only, and is not limited by

any a priori assumptions about criterion distributions. It satisfies the conditions of a joint

probability distribution identified in Chapter IV, since its actual distribution function is

derived from the univariate distribution. It does not require any (numerical) integration

or rely on any particular sampling method and can be used as long as sample data is

available. The need for this data, however, is its very limitation, since it can only be used

in a design process with available simulation/modeling or test data. Given enough

sample data, however, the joint EDF yields the most accurate joint distribution

prediction, since it does not rely on any approximation to generate criterion statistics. Its

greatest advantage is that it does not require a correlation coefficient, which can be

difficult to estimate reliably in a design process. For very large numbers of sample data,

the joint EDF can yield the exact solution for the joint distribution.


Joint Probability Model:

A joint probability model is an explicit formulation of a parametric joint

probability density (or cumulative) distribution function that can be used as an algorithm

to compute the joint probability. Its key characteristic is that it is not based on sample

data, like the Empirical Distribution Function, but rather the statistics or characteristics of

parametric univariate criterion distributions. This characteristic is important, since it

allows for the continued use of the probabilistic information generated by the traditional

probabilistic design process with its univariate criterion distribution output.

The few explicit formulations of a joint probability density function, published in

the literature, are based on the joint Normal-Distribution, the two main references being

[Garvey, 1999] and [Tong, 1990]. Garvey’s book also includes the bivariate Normal,

Normal-Lognormal, and Lognormal joint distributions, published first in [Garvey, Taub,

1992]. For illustration purposes, the bivariate Normal-Distribution is reproduced here:

−+−−−

−−−

=22

222

22

1exp

12

1),(

Y

Y

Y

Y

X

X

X

X

YX

XY

yyxxyxf

σµ

σµ

σµρ

σµ

ρρσπσ. (5.5)

Note that the only information needed for this model consists of the means µX and

µY, the standard deviations σX and σY, and the correlation coefficient ρ for the criteria X

and Y. The model variables, x and y, are defined over the interval of all possible criterion

values. The advantage of this model is the limited information needed, which makes it

very flexible for use and application. For example, if only expert knowledge and no

simulation/modeling is available in the early stages of design or technology development,

educated guesses for the means, standard deviations, and correlation coefficients can be


used to execute the joint probability model. Unfortunately, it is only valid for two

variables at a time.

While Tong’s book does not include Garvey’s Lognormal and Normal-Lognormal

formulations, it does extend the bivariate joint Normal formulation to N dimensions

[Tong, 1990]:

Definition: An N-dimensional random vector10 X with mean vector µ and

covariance matrix Σ is said to have a nonsingular multivariate Normal-Distribution,

Σ > 0, if (i) Σ is positive definite, and (ii) the density function of X is of the form

2/),;(2/12/)2(

1),;( xx NQ

Nef −=

πand Nℜ∈x (5.6)

where )()(),;( 1 xxx −′−= −NQ and =

22211

22222112

1121122

1

NNNNN

Nn

Nn

σσσρσσρ

σσρσσσρσσρσσρσ

L

MOMM

L

L

. (5.7)

X is said to have a singular multivariate Normal-Distribution (X ~ NN(µ, Σ), |Σ| = 0), if (i)

Σ is positive semidefinite, and (ii) for some r < N there exists an N x r real matrix C such

that X and CZr + µ are identically distributed, where Zr ~ Nr(0, Ir).

While the bivariate density functions are valid for the entire range of the

correlation coefficient ρ, the multivariate Normal-Distribution is not defined for certain

negative ρ. Specifically, condition (i), Σ being positive (semi-)definite, is violated for

certain ρ, which can easily been seen from the following proof.

10 Vector of random variables.


Proof: Without loss of generality, assume all standard deviations σi are equal to

one, and all correlation coefficients are equal and have a value -1 < ρ < 1. In this case, Σ

becomes a very simple symmetric matrix of the form:

=

1

1

1

L

MOMM

L

L

ρρ

ρρρρ

. (5.8)

In order for Σ to be positive (semi-)definite, all of its eigenvalues need to be

positive (or zero).[Strang, 1993] The eigenvalues λ can be determined from [Strang,

1993]:

=

−

−−

⇔=−

0

0

0

1

1

1

0 2

1

)(MM

L

MOMM

L

L

Nx

x

x

λρρ

ρλρρρλ

λ xI (5.9)

which yields a system of linear equations with N-1 equations of the form

solution) zero-nononly (the 1

0)1()1()1( 21

ρλρλρλρλ

−=⇒=−−++−−+−− Nxxx L

(5.10)

and one equation of the form

( ) ( ) ( )solution). zero-nononly (the )1(1

0)1(1)1(1)1(1 21

ρλρλρλρλ

−+=⇒=−+−++−+−+−+−

N

xNxNxN NL(5.11)

While the N-1 eigenvalue solutions in Equation 5.10 are all positive for

-1 < ρ < 1, the solution for λ in Equation 5.11 is negative for ρ < 1/(1-N). Hence, for

correlation coefficient values ρ < 1/(1-N), Σ is not positive (semi-)definite and the joint

normal density function is not defined.


A similar proof can be made for other standard deviation and correlation

coefficient values. Some values for σi and ρij may produce a positive (semi-)definite Σ,

while others do not. The conclusion to be made is that the algorithm in Equation 5.6 is

not valid for all ρij ∈ (-1,1), and is hence unsuitable for a generally applicable joint

probabilistic decision making technique. Finally, the integral of the characteristic

function for a univariate Normal-Distribution does not exist and consequently neither

does the one for Equations 5.5 and 5.6. As a result, a numerical integration is needed to

determine the cumulative probability for both algorithms. While the bivariate algorithms

listed in [Garvey, 1999] are simple enough to be integrated numerically, Equation 5.6

accommodates many variables and its numerical integration over a large number of

criteria becomes too extensive of a task. Consequently, algorithms based on a joint

Normal-Distribution are considered unsuitable for systems design and other algorithms

need to be identified.

New Algorithm:

To remedy the shortcomings of the existing algorithms and to allow the use of

non-gaussian joint probability distributions, a new algorithm is proposed for this thesis

work. Any function that satisfies the conditions in Equations 4.3 and 4.4 can be

classified as a continuous joint probability density function. It can further be assumed

that a suitable joint density function entails the product of the univariate probability

distributions for all random variables. This is particularly true for independent random

variables (Equation 4.10), and is also reflected in the models in [Garvey, 1999] and


[Tong, 1990]. Without loss of generality, it can further be assumed that a correlation

function, accounting for the correlation between random variables, can be multiplied with

this product of univariate density functions to generate the new joint probability density

function. Explicitly:

( )NXXXN xxxgfffC

xxxfN

,....,,1

),....,,( 2121 21⋅⋅= L (5.12)

with ( )∫∫ ⋅⋅=Ω

NNXXX dxdxdxxxxgfffCN

LLL 2121 ,....,,21

. (5.13)

f satisfies the conditions in Equations 4.3 and 4.4 as long as the correlation

function g satisfies these conditions. Specifically, g has to satisfy the following

requirements in order for f to be a joint probability density function:

• g ≥ 0, since 0),....,,( 21 ≥Nxxxf and 0,,,21

≥NXXX fff K . (5.14)

• g(x1, x2, …, xN) = 1, when X1, …, XN are independent, sinceNXXX ffff L

21⋅= .(5.15)

The second condition in Equation 4.3 is always satisfied by f, independent of g, since

( ) 1,....,,1

....),....,,( 21212121 21=⋅⋅=⋅⋅⋅ ∫∫∫ ∫

ΩΩNNXXXNN dxdxdxxxxgfff

Cdxdxdxxxxf

NLLL

with ( )∫∫ ⋅⋅=Ω

NNXXX dxdxdxxxxgfffCN

LLL 2121 ,....,,21

.

Using Equation 5.12 as an algorithm for the determination of the joint probability

reduces the problem of finding a suitable joint probability density function to finding a

suitable correlation function. In general, all correlation functions, satisfying the

conditions in Equations 5.14 and 5.15, are possible functions, while some are easier to

implement in a computer algorithm than others.


The fundamental problem for the determination of the joint cumulative

probability is the problem of dimensionality that prohibits a numerical integration in the

case of four or more criteria (at current processor speeds). If a hundred integration points

are desired in each dimension for sufficient accuracy, four criteria would require

(102)4 = 108 = 100 million function evaluations, a prohibitively expensive task in systems

design. An alternative approach is to integrate the joint density function analytically,

since the univariate density functions and the correlation function are explicitly known.

This approach however puts an additional restriction on the correlation function and the

univariate PDFs:

∫Ω

⋅ iNiiX dxxxxxgxfi

),,,,()( 21 KK , i = 1, 2, …, N, (5.16)

needs to be defined for all criteria i. This requirement can be derived from simple

integration rules for independent functions:

( )∫ ∫∫∫∫ ∫ ⋅=⋅⋅⋅ΩΩ

12212121 ,....,,1

....),....,,(21

dxdxdxxxxgfffC

dxdxdxxxxf NNXXXNN NLL .(5.17)

Hence, each integration step in Equation 5.17 requires ( )∫Ω

⋅ iiX dxxgfi

to exist and

be finite. This, in turn, becomes a requirement for the correlation function, once the

univariate distribution function has been chosen for Xi. A simple example for g is the

linear function

∑ ∑−

= +=

−−+

=

1

1 121 2

),,,(N

i

N

ij j

jj

i

iiijN range

medianx

range

medianxNxxxg ρK . (5.18)


The transformation for xi, using rangei and mediani, guaranties that g > 0. The

variable rangei denotes the length of half the range in which the distribution of Xi has

sufficiently large probability/frequency values, i.e. rangei = (Maxi – Mini)/2. The term

“sufficiently large” is at the user’s discretion, making rangei an input to the algorithm.

By applying Equation 5.18, Equation 5.16 can be simplified using the product rule:

( ) ∫∫ ∂∂−⋅=⋅

Ω

max

min

max

min

)(][

i

i

i

i

iii

x

x

iXi

ixxXiiX dtF

t

tggFdxxgf . (5.19)

If i

i

t

tg

∂∂ )(

is not a function of ti, i.e. g is linear with respect to ti, the integral

( )∫Ω

⋅ iiX dxxgfi

exists and is finite for all univariate probability distributions that can be

integrated twice. A more comprehensive list of suitable correlation functions for this new

algorithm is provided in Appendix B.

The following bivariate example illustrates the new algorithm and the use of the

correlation function introduced in Equation 5.18. Assume random variables X and Y are

distributed according to a Gamma-Distribution, X ~ G(4,3,5), Figure 5.3, and a Weibull-

Distribution, Y ~ W(3,2,7), Figure 5.4, respectively.11 Further assume that both variables

are correlated with a correlation coefficient ρ = -0.6. Consequently, the proposed joint

probability function is formulated as:

444 3444 21444 3444 21

44 344 21),(

)(

)7(312

)(

3

5

4

14

1

8

20

256.01)7(23

)4(3

)5(),(

2

yxgyf

y

xf

x yxeye

xyxf

Y

X

−−−⋅−⋅⋅

Γ−= −−−

−−−

. (5.20)

11 See Appendix A for a definition and description of the distributions used in this thesis.


0 10 20 30 40 500

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

x

Figure 5.3: Gamma-Distribution for X

7 7.5 8 8.5 90

0.5

1

1.5

y

Figure 5.4: Weibull-Distribution for Y

The graphs in Figures 5.5 and 5.6 display the joint probability distribution f(x,y) in

three and two dimensions, respectively. In order to be able to display the distribution

better, the graph in Figure 5.5 is rotated 180° around the center axis. The graph in

Figure 5.6 displays concentric lines of constant frequency indicating the location of the

area of highest frequency around the common center. The lines are obtained by slicing

the three-dimensional distribution at several levels of constant frequency and projecting

the cut onto the two-dimensional plane spanned by the two random variables.

Figure 5.5: Example Joint ProbabilityDistribution in 3D

10 20 30 407

7.5

8

8.5

9

x

y

Figure 5.6: Contour Plot of ExampleJoint Probability Distribution

Increasing Frequency

Pro

babi

lity

Den

sity

Pro

babi

lity

Den

sity


Five Schemes for Implementation

In order to be able to use Equations 5.3 and 5.12 for probabilistic systems design,

schemes for practical implementation need to be established. Five such schemes are

introduced in this thesis, all of which are extensions of the three probabilistic design

methods outlined in Chapter III. The schemes are summarized in Figure 5.7, indicating

that all assume some modeling and simulation, either in form of computer programs or

explicit equations.

Problem Definition Modeling andCriteria Simulation

Problem Definition Modeling andCriteria Simulation

Monte-CarloSimulation

EmpiricalDistribution

Function

JointProbability

Model

JointProbability

Model

JointProbability

Model

Metamodel

Fast ProbabilityIntegration

CDFRegression

CDFRegression

CorrelationFunctionMonte-Carlo

Simulation

CorrelationFunction

Scheme #I Scheme #II Scheme #V

LeastEffort

MostAccurate

CDFRegression

CorrelationFunction

EmpiricalDistribution

Function

Scheme #III Scheme #IV

Figure 5.7: Five Schemes for the Evaluation of the Joint Probability Distribution

In order to illustrate the five different schemes, the example problem utilized in

Chapters II and III is employed to generate a joint probability distribution with each


scheme, followed by a comparison of all distributions. As noted in Chapter III, the

values for y1 and y2 are assumed to be uncertain and normal distributions are assigned to

them, such that Y1 ~ N(1,1) and Y2 ~ N(1,1). Also, the deterministic variables are held

constant for each distribution, such that x1 = x2 = 1.

Scheme #I – Monte-Carlo Simulation with Empirical Distribution Function

The first scheme is an extension of the probabilistic design Method #1, identified

in Chapter III. It uses the samples from the system analysis tool generated through a

Monte-Carlo simulation as data for the Empirical Distribution Function (Equation 5.3).

This method is the most accurate, but requires the most system analysis effort, which can

be computationally expensive.

To create the distribution based on the example problem from Chapters II and III,

a Monte-Carlo simulation is employed to generate 10,000 sample values from the noise

variable distributions for Y1 and Y2 as inputs to Equations 2.16 and 2.17. The resulting

10,000 samples for F1 and F2 are then used in the Empirical Distribution Function

(Equation 5.3):

( )∑=

===M

iM fFfFIfff

12211

121 ,),( , (5.21)

where ( )

+≤≤−∧+≤≤−===

otherwise02222

for 1,2

222

21

111

12211

εεεεfFffFffFfFI .

ε1 and ε2 denote the ranges around f1 and f2 that F1 and F2 must fall within,

respectively. Equation 5.21 can be used to generate the plots of the joint probability

distribution displayed in Figures 5.8 and 5.9. Note the sharp frontier in the two-


dimensional representation of the joint probability distribution in Figure 5.9. It is due to

the local (and global) optima of f1 and f2 within the distribution range of Y1 and Y2. The

scattered lines in this plot are contours of equal probability density, produced by the

sample points that are found throughout the f1-f2-plane in this scattered fashion.

Figure 5.8: Example DistributionGenerated by Scheme #I (3D)

-10 -5 0 5 10

0

5

10

15

20

25

30

f1

f2

Figure 5.9: Example DistributionGenerated by Scheme #I (2D)

Scheme #II – Monte-Carlo Simulation with Joint Probability Model

Scheme #II is also an extension of the probabilistic design Method #1 from

Chapter III, regressing the data generated by the same Monte-Carlo simulation for each

criterion individually in order to obtain its univariate probability distribution function. In

addition, a correlation function is determined, e.g. via correlation coefficients for each

pair of criteria. The joint probability model in Equation 5.12 can then be used with the

univariate distributions and the correlation function to create the joint probability

distribution. Due to the regression of the probability distributions, this scheme produces

a joint probability distribution less accurate than the one in Scheme #I, but it yields

graphs that are more amenable (no scatter plots) and therefore to be preferred for plotting.

-10-5

50

10


To create the joint probability distribution with Scheme #II, the 10,000 samples

from Scheme #I are regressed to establish univariate probability distributions for F1 and

F2. The regression is illustrated in Figure 5.10 for F1 and Figure 5.11 for F2, yielding a

Beta-Distribution with α = 17, β = 7, Min = -10, Max = 11.46 for F1 and a Gamma-

Distribution with parameters a = 4, b = 2, Loc = -4.31 for F2. The correlation coefficient

for F1 and F2 was found to be ρ = 0.08.

.000

.016

.032

.048

.064

Figure 5.10: Distribution Regression for F1 in Scheme #II

.000

.026

.052

.077

.103

Figure 5.11: Distribution Regression for F2 in Scheme #II

Once the regression has determined the correlation coefficient and the univariate

probability distributions for F1 and F2, the joint probability distribution can be

determined based on the Joint Probability Model in Equation 5.12 with a correlation

function based on Equation 5.18:

Freq

uenc

y Beta-Distribution

F1

11.56.51.5-3.5-8.5 f1

Freq

uenc

y Gamma-Distribution

F2

3525155-5 f2


4444 34444 21444 3444 214444444 34444444 21),(

21

)(

4

2

31.414

2

)(

17

1

117

121

2122

2

11

2012

73.1073.0

08.01)4(2

)31.4(46.21

46.1146.2110

)7()17()717(

),(

ffgff

f

ff

ffeffffff

FF

−−−⋅

Γ+⋅

−

+

ΓΓ+Γ=

+−−−−

. (5.22)

The joint probability distribution is visualized in Figures 5.12 and 5.13 for three

and two dimensions respectively. Note that the contours of constant density in

Figure 5.13 are much smoother than in Figure 5.9.

Figure 5.12: Example DistributionGenerated by Scheme #II (3D)

-5 0 5 10-5

0

5

10

15

f1

f2

Figure 5.13: Example DistributionGenerated by Scheme #II (2D)

Scheme #III – Metamodel/Monte-Carlo Simulation with Empirical Distribution Function

Scheme #III is an extension of the probabilistic design Method #2, using samples

for the Empirical Distribution Function (Equation 5.3), obtained from a metamodel and

generated by a Monte-Carlo simulation. This method may require significantly less

simulation analysis to generate the metamodel, but its accuracy in predicting the joint

probability distribution depends heavily on the prediction accuracy of the metamodel

itself. To create the distribution based on the example problem from Chapters II and III,

Equations 2.16 and 2.17 are approximated with a response surface equation (RSE). As


pointed out in the example for Chapter III, while f1 is a quadratic polynomial already and

can be used directly as an RSE. f2 on the other hand can be approximated, with Equation

3.8 as the corresponding response surface equation. Subsequently, a Monte-Carlo

simulation is employed to generate 10,000 sample values from the noise variable

distributions for Y1 and Y2 as inputs to Equations 2.16 and 3.8. The resulting 10,000

samples for F1 and F2RSE are then used in the Empirical Distribution Function

(Equation 5.3):

( )∑=

===M

i

RSEM fFfFIfff

1211

121 2

,),( , (5.23)

where ( )

+≤≤−∧+≤≤−===

otherwise02222

for 1,2

22

21

111

1211

22

εεεεfFffFffFfFI

RSERSE .

ε1 and ε2 denote the ranges around f1 and f2 that F1 and F2RSE must fall within,

respectively. Equation 5.23 can be used to generate the plots of the joint probability

distribution displayed in Figures 5.14 and 5.15. Note again the sharp frontier in the two-

dimensional representation of the joint probability distribution in Figure 5.15, which is

due to the local (and global) optima of f1 and f2RSE within the distribution range of Y1 and

Y2. The scattered lines in this plot are contours of equal probability density, produced by

the sample points that are found throughout the f1-f2-plane in this scattered fashion.


Figure 5.14: Example DistributionGenerated by Scheme #III (3D)

-5 0 5 10

5

10

15

20

f1

f2

Figure 5.15: Example DistributionGenerated by Scheme #III (2D)

Scheme #IV – Metamodel/Monte-Carlo Simulation with Joint Probability Model

Scheme #IV is also an extension of the probabilistic design Method #2, identified

in Chapter III. In this case, the data generated by the Monte-Carlo simulation is based on

a metamodel as system analysis code approximation and regressed for each criterion

individually to obtain its univariate probability distribution. The information for the

correlation function, e.g. correlation coefficients for each pair of criteria, of the Joint

Probability Model (Equation 5.12) can be obtained from either the system analysis data

used for the generation of the metamodel, or the Monte-Carlo simulation data based on

the metamodel. When the metamodel generation calls only for few system analysis

simulations, the Monte-Carlo simulation data provides the greater statistical significance

for the correlation coefficients and is to be preferred, despite the fact that the data is

based on a system analysis code approximation. When the metamodel generation

requires a significant amount of system analysis already, the data might as well be used

-55

010


for the correlation coefficient estimation, since it is more accurate than the Monte-Carlo

simulation data. This scheme is less accurate than Scheme #III, but produces the nicer

graphs, similar to Scheme #II, and is to be preferred for plotting. To create the joint

probability distribution with Scheme #IV, the 10,000 samples from Scheme #III are

regressed to establish univariate probability distributions for F1 and F2. Note that these

samples are based on the metamodels for f1 and f2, Equations 2.16 and 3.8. The

regression is illustrated in Figure 5.16 for F1 and Figure 5.17 for F2, yielding a Beta-

Distribution with α = 17, β = 7, Min = -10, Max = 11.46 for F1 and a Gamma-

Distribution with parameters a = 4, b = 1, Loc = 0.78 for F2. The correlation coefficient

for F1 and F2 was found to be ρ = 0.02.

.000

.016

.032

.048

.064

Figure 5.16: Distribution Regression for F1 in Scheme #IV

.000

.015

.030

.045

.059

Figure 5.17: Distribution Regression for F2 in Scheme #IV

Freq

uenc

y Beta-Distribution

F1

11.56.51.5-3.5-8.5 f1

Freq

uenc

y Gamma-Distribution

F2

20151050 f2


Once the regression has determined the correlation coefficient and the univariate

probability distributions for F1 and F2, the joint probability distribution can be

determined based on the Joint Probability Model in Equation 5.12 with a correlation

function based on Equation 5.18:

44444 344444 21444 3444 214444444 34444444 21),(

21

)(

4

1

78.014

2

)(

17

1

117

121

2122

2

11

10

10

73.10

73.002.01

)4(1

)78.0(

46.21

46.11

46.21

10

)7()17(

)717(),(

ffgff

f

ff

ffeffffff

FF

−−−⋅

Γ−⋅

−

+

ΓΓ+Γ=

−−−−−

. (5.24)

The joint probability distribution is visualized in Figures 5.18 and 5.19 for three

and two dimensions respectively. Note that the contours of constant density in

Figure 5.19 are much smoother than in Figure 5.15.

Figure 5.18: Example DistributionGenerated by Scheme #IV (3D)

-5 0 5 10-5

0

5

10

15

f1

f2

Figure 5.19: Example DistributionGenerated by Scheme #IV (2D)

Scheme #V – Fast Probability Integration with Joint Probability Model

The fifth scheme is the only one identified here that extends the probabilistic

design Method #3. It uses a fast probability integration technique, like the Advanced

Mean Value (AMV) method, or one of its derivatives, to obtain a univariate cumulative


distribution function for each criterion, which subsequently can be used in the Joint

Probability Model (Equation 5.12). A disadvantage of this particular scheme, however, is

its lack of correlation information on the criteria to create the correlation function for the

algorithm. This means that the correlation function must be obtained through some other

means. Also, in the case of the AMV method, the scheme only yields one univariate

CDF per probabilistic analysis. Therefore, despite the high efficiency of the AMV

method, Scheme #V can become computationally intensive for large numbers of criteria.

Last but not least, the transformation of the CDF into a probability function that can be

used in Equation 5.12 is not a trivial task, since the information on the CDF is generated

in form of function values only. The accuracy of Scheme #V, using the AMV method, is

similar to the accuracy of Scheme #IV with a response surface equation, since the AMV

method has a prediction accuracy similar to a response surface equation12 and both

methods rely on a probability distribution regression.

In order to generate the joint probability distribution for the example problem in

Chapters II and III, the univariate distributions generated by the AMV method in the

example for Chapter III can be employed directly. By visual comparison of the

cumulative probability plot (Figure 3.11), the distribution for F1 is determined the to be a

Beta-Distribution with α = 10, β = 5, Min = -10, Max = 10. Based on the cumulative

distribution plot in Figure 3.12, the univariate distribution for F2, estimated with the

AMV method, is very similar to the distribution based on the Monte-Carlo simulation.

12 See Chapter III for discussion.


For this reason, F2 in Scheme #V is assumed to have the same distribution as in

Scheme #II, a Gamma-Distribution with parameters a = 4, b = 2, and Loc = -4.31. The

correlation coefficient cannot be determined with Scheme #V, but is borrowed for this

example from Scheme #II: ρ = 0.08.

.000

.026

.052

.077

.103

Figure 5.20: Distribution Regression for F2 in Scheme #V

With the correlation coefficient and the univariate probability distributions for F1

and F2, the joint probability distribution can be determined based on the Joint Probability

Model in Equation 5.12 with a correlation function based on Equation 5.18:

4444 34444 21444 3444 21444444 3444444 21),(

21

)(

4

2

31.414

2

)(

15

1

110

121

2122

2

11

20

12

20

008.01

)4(2

)31.4(

20

10

20

10

)5()10()510(

),(

ffgff

f

ff

ffeffffff

FF

−−−⋅

Γ+⋅

−

+

ΓΓ+Γ=

+−−−−

. (5.25)

The joint probability distribution is visualized in Figures 5.21 and 5.22 for three and two

dimensions respectively.

Freq

uenc

y GammaDistribution

F2

3525155-5 f2


Figure 5.21: Example DistributionGenerated by Scheme #V (3D)

-5 0 5 10-5

0

5

10

15

f1

f2

Figure 5.22: Example DistributionGenerated by Scheme #V (2D)

Comparison of Schemes

In order to compare the relative performance of the five schemes, three plots are

employed to provide a visual contrast of the different joint probability distributions. The

first two plots in Figures 5.23 and 5.24 compare the joint distribution from Scheme #I

with the one from Scheme #II and the one from Scheme #III with the one from

Scheme #IV respectively. Essentially this comparison tests the ability of the regression

in Scheme #II and Scheme #IV together with the correlation function to accurately

represent the data from the Monte-Carlo simulation.

Both plots clearly indicate that Schemes #II and #IV are not capturing the steep

ridge of the distributions from Schemes #I and #III. Both distributions yield a certain

likelihood for high values for f1 and low values for f2, which the data does not indicate. It

appears that particularly the correlation function in Equation 5.18 is not adequate for the

presented data, since the correlation function used is symmetric while the data is

asymmetric. Unfortunately, the right correlation function can only be determined after


the Monte-Carlo simulation data has been generated so that Equation 5.18 with its

correlation coefficients is still the best first guess when more detailed information is

lacking. Also, the distributions from Schemes #II and #IV are both yielding the right

location of the distribution so that both can be used in visual comparisons of joint

probability distributions for different products. However, for the estimation of an

accurate joint probability value Schemes #I and #III are to be preferred over Schemes #II

and #IV.

-5 0 5 10-5

0

5

10

15

20

f1

f2

Figure 5.23: Comparison of JointDistributions from Schemes #I and #II

-5 0 5 100

5

10

15

20

f1

f2

Figure 5.24: Comparison of JointDistributions from Schemes #III and #IV

The plot in Figure 5.25 finally compares the three schemes employing the Joint

Probability Model in Equation 5.12. Here the differences are entirely due to the different

univariate distributions, regressed from system analysis data (Scheme #II), regressed

from response surface equation data (Scheme #IV), or generated through the AMV

method (Scheme #V). The distribution from Scheme #II can be seen as the most

accurate, despite the aforementioned criticism, with the distribution from Scheme #V as a


close second best. Scheme #IV seems to miss the target distribution the most, due to the

prediction inaccuracy of the response surface equation. Note that these results cannot be

generalized, since Scheme #IV may yield a distribution quite accurate, if the metamodel

prediction is accurate. This is particularly true for aerospace systems design, where

response surface equations have demonstrated their adequate approximation capability of

the economic objectives specifically.[Mavris, Bandte, 1998], [Mavris, Bandte,

DeLaurentis, 1999], [Mavris, Bandte, Schrage, 1995]

-5 0 5 10-5

0

5

10

15

f1

f2

Figure 5.25: Comparison of Joint Distributions from Schemes #II, #IV and #V

Joint Probabilistic Decision Making Technique

Once the joint probability distribution has been obtained through one of the five

schemes identified in the previous section, the joint Probability of Success (POS), can be

determined. The joint Probability of Success is the envelope objective function or overall

evaluation criterion for the Joint Probabilistic Decision Making (JPDM) technique,

measuring the probability of satisfying all criteria. The key concept in this technique is

an area of interest, limited by the values zjmin and zjmax for each criterion j, that contains

Scheme #II

Scheme #IV

Scheme #V


all values the decision-maker deems satisfactory. The joint probability distribution is

then superposed with this area of interest, yielding the Probability of Success as the

volume underneath the distribution, over the area of success. This concept is visualized

in Figure 5.26 and mathematically represented by:

∑=

≤≤=M

jjjj zzzI

MPOS

1maxmin )(

1, (5.26)

using the Empirical Distribution Function (Equation 5.3), while M are the number of

samples, and

∫=max

min

)(z

z

tt dfPOS , (5.27)

using the Joint Probability Model (Equation 5.12).

0.0 00

0 .001

0 .0 02

0 .00 3

0 .0 04

0 .00 5

0 .0 06

Pro

ba

bil

ity

De

ns

ity

z1max

Ellipses of Constant Density

z1min

z2min

z2max

z1

z2

z1max

z1min

z2min z2max

z1

z2

Area ofInterest

Figure 5.26: Display of the Joint Probability Distribution for Two Criteria

An information flowchart of the Joint Probabilistic Decision Making technique is

displayed in Figure 5.27, arriving at the joint Probability of Success through a process

that contains eight elements: identification of criteria, assignment of probability


distributions to noise variables, fixation of control variables, identification of the system

analysis tool, evaluation of the joint probability distribution, establishment of criterion

values, determination of preferences among criteria, and calculation of the joint

Probability of Success.

Criteria (Z)Objectives/Requirements

1

Analysis ToolZ = f(x,Y)

CriterionValues

(zmin, zmax)6Weights (w)

7

4

AlternativesBaseline

Uncontrollable/Noise Variables

YNon-Deterministic

ControlVariables

x = (0.5, 4, 0.07, …)

Deterministic

Joint ProbabilityDistribution

(Five Schemes)

POS

2

3

5

8

Figure 5.27: Joint Probabilistic Decision Making Technique

Element 1 – Identification of Criteria

First, the criteria for the decision making process need to be determined. They are

typically comprised of customer requirements or desirements as well as objectives that

need to be satisfied from the designer’s perspective. These criteria are usually

established in the early conceptual design phase of the product (see Chapter I). The Joint

Probabilistic Decision Making technique treats this set of criteria as a random vector,

represented by Z.


Element 2 – Assignment of Probability Distributions to Noise Variables

All variables that are not under the control of the designer, i.e. their values are not

known with certainty, need to be assigned probability distributions that represent the

likelihood of taking on certain values. This element allows the subsequent simulation to

evaluate a range of values rather than a single deterministic number.

Element 3 – Fixation of Control Variables

For each probabilistic analysis, all variables that are part of the analysis and under

the control of the designer13 need to be held constant. For a product selection, i.e.

evaluation of alternatives, the vector x, containing the control variable values, is a

representation of each alternative under consideration. For optimization, the vector x is

the set of design variables that is manipulated by the optimization routine.

Element 4 – Identification of System Analysis Tools

Chapter I underlined the importance of mathematical analysis in systems design,

for example in the form of a computer program. For this reason, the probabilistic design

methods in Chapter III and the five implementation schemes from the previous section all

center around system analysis on a computer. Since the system analysis tool is of such

importance, the following principles for the selection of its tools should be kept in mind:

• the analysis tool needs to yield numerical information on the criteria, and

• the numerical information should be dependent on the noise and control variables.

13 A variable is under the control of a designer if its value is considered known (with certainty).


Element 5 – Evaluation of the Joint Probability Distribution

This element is comprised of one of the five schemes for calculating the joint

probability distribution, outlined in the previous section and Figure 5.7. It is the key

element of the Joint Probabilistic Decision Making technique and is the major scientific

contribution of this thesis work.

Element 6 – Establishment of Criterion Values

As outlined in the beginning of this section, the Probability of Success is

determined by two components: the joint probability distribution and the area of interest.

The values defining the area of interest are a minimum and a maximum14 for each

criterion and are listed in the vectors zmin and zmax. Values in between these limits are

considered to be satisfying the decision-maker’s objective. Typically, these values are

supplied through Requests for Proposals or other means during the initial design phase.

Additional requirements may be imposed through design and performance goals,

government regulations, or customer desirements.

Element 7 – Determination of Preferences Among Criteria

The seventh element of the JPDM technique determines preferences supplied by

the customer or designer for each criterion. Preferences indicate which criteria are more

important than others to the decision-maker. They are usually represented by a set, or

vector w, of (preference) ‘weights’ which are normalized to sum to 1, signifying the

14 infimum and supremum in the case of minus and plus infinity


relative importance of each criterion.[Hwang, 1981] If no criterion is associated with a

prevalent preference over other criteria, all weights wi, i = 1, 2, …, N, are assigned an

equal value of 1/N, with N being the number of criteria.

Element 8 – Calculation of the Probability of Success

The eighth element finally combines the criterion values, the weights, and the

joint probability distribution function to calculate the joint Probability of Success (POS).

The Probability of Success denotes the chance for a design solution to produce criterion

values within the area of interest and constitutes the objective function or evaluation

criterion for the Joint Probabilistic Decision Making technique. POS is always attempted

to be maximized, while values larger than one are impossible.

While this section described the essence of the Joint Probabilistic Decision

Making technique itself, the following sections outline its use and application,

particularly for optimization, product selection, and requirement trade-off.

Optimization

One important use of the Joint Probabilistic Decision Making technique is for

optimization. As outlined in Chapter II, current multi-criteria optimization techniques

suffer from an inability to account for uncertain parameters. As remarked in Chapter III,

current probabilistic design methods only account for one criterion at a time without

supplying information about the likelihood of meeting other criteria. The use of the joint

Probability of Success (POS) as an objective function for optimization, accounts for both,

uncertainty in the analysis parameter values and a multitude of criteria. Principally, the


multi-criteria optimization problem is transformed into a univariate maximization

problem, trying to achieve the highest possible Probability of Success by perturbing the

design variables. As such, the baseline values for the design variables may situate the

joint probability function as indicated in Figure 5.28, yielding a small Probability of

Success. By manipulating the design variables, however, the joint probability

distribution may be shifted into the area of interest, yielding a higher Probability of

Success. Naturally, POS cannot take on values higher than one. The Joint Probabilistic

Decision Making technique in itself does not provide an optimization scheme, but POS

can be used as an objective function for any univariate optimization scheme. Examples

of such schemes can be found in [Reklaitis, Ravindran, Ragsdell, 1983] and

[Vanderplaats, 1999]. To illustrate the use of the Joint Probabilistic Decision Making

technique for optimization, it is extended in Figure 5.29 to include the optimization loop

and an exit criterion.

Criterion 1

Cri

teri

on 2

Optimize

Baseline

Optimal

Figure 5.28: Shifting the Joint Probability Distribution During Optimization

Area ofInterest


Weights (w)


7

4

Baseline


YNon-Deterministic

ControlVariables

x = (0.5, 4, 0.07, …)

Deterministic


(Five Schemes)

POS

2

3

5

UpdateDeterministicVariables inOptimization

Scheme

SatisfactoryPOS ?

Solution withHighest POS

Y

N


1

9

Optimization Loop

8

10

CriterionValues

(zmin, zmax)6

Figure 5.29: Joint Probabilistic Decision Making Technique for Optimization

First the criteria are identified based on which the decision is made, i.e. based on

which the design is evaluated. Second, probability distributions are assigned to the noise

variables, i.e. the variables that introduce the uncertainty into the optimization process.

Third, a baseline is established that corresponds to a set of control variable values. Next,


the joint probability distribution is evaluated through the use of one of the

aforementioned five schemes and the system analysis tool. Finally, using the weights and

criterion values that identify the area of interest, the Probability of Success is calculated

for the baseline.

The new Element #9 is a checkpoint that determines whether the calculated POS

is large enough. If so, the current setting of design variables represents the best solution.

If the baseline, or starting point, already yields a probability of one, which is unusual for

a baseline, the criterion values should be revisited and more realistic values should be

established. If POS is not deemed satisfactory, the deterministic control variables need to

be updated based on the optimization scheme employed (Element #10).

Using the updated values for the deterministic variables, the probabilistic analysis

is repeated to yield the new POS, thereby closing the optimization loop. Note that the

noise variable distributions, criterion values, and weights, do not change during the entire

optimization process, only the deterministic variables change from one iteration to the

next. Once POS cannot be improved upon, the optimization loop is terminated and the

last design variable setting constitutes the solution. To illustrate the use of the Joint

Probabilistic Decision Making technique for optimization further and compare it to

existing techniques, the example from Chapters II and III is employed here again.


To demonstrate the use of the Joint Probabilistic Decision Making (JPDM)

technique for optimization, the example previously used to generate joint probability

distributions for the five different schemes is employed here again. Furthermore, to


demonstrate its capability, optimization with JPDM is compared to the methods used in

the example of Chapter II: MaxiMin, Overall Evaluation Criterion, and Goal Attainment

method.

To illustrate the process outlined in Figure 5.29, the example problem is first

mapped to all elements of the process. The obvious starting point is the identification of

the criteria: f1 and f2. As before, the values for y1 and y2 are assumed to be uncertain and

normal distributions are assigned to them, such that Y1 ~ N(1,1) and Y2 ~ N(1,1). The

values for the deterministic variables are selected for each optimization loop based on a

simplex or sequential programming/linear search algorithm, with a baseline or initial

guess of x1 = x2 = 1. The “system analysis tool” is comprised of two example equations:

212

222

111 )(5.0)(5.05 YYYxYxF +−−−−= (5.28)

2221

212 coscos

2YxxYYF +++−= (5.29)

subject to: 22 1 ≤≤− x and 22 2 ≤≤− x .

To evaluate the joint probability distribution, Scheme #I suggests itself, since it is

the most accurate scheme and the system analysis is simple enough to generate an

extensive amount of data directly from it. Also, no plotting of joint probability

distributions is necessary at this point. The criterion values to be satisfied have been

established in earlier uses of the example problem to be F1 ≥ 5 and F2 ≥ 5. The important

difference between this example and that in Chapter III is that POS represents the

probability of both, F1 and F2, being larger than 5 concurrently. Therefore, the

optimization problem can be formulated as:


)5coscos , 5)(5.0)(5.05(max

)5 , 5(maxmax

2221

2121

222

211

,

21,,

221

2121

≥+++−≥+−−−−

=≥≥=

YxxYYYYYxYxP

FFPPOS

xx

xxxx(5.30)

subject to: 22 1 ≤≤− x , 22 2 ≤≤− x , Y1 ~ N(1,1), and Y2 ~ N(1,1).

Using this formulation and the process described above, POS is calculated for the

baseline to be 0.1254. This Probability of Success can certainly be improved upon and

the deterministic variables are adjusted by the optimization schemes, thereby entering a

new iteration in the optimization loop. Once no improvement in POS can be found

through adjustment of the deterministic variables, the solution to the optimization

problem is established with the last setting of the deterministic variables.

To illustrate the use of Equation 5.30 as an objective function for optimization

and in order to get a better feel for the nature of the objective function in this example, a

surface plot is generated over the design space and displayed in Figure 5.30. Note that,

similar to the surface plot in Figure 3.16, POS has a steep drop off and a somewhat

rugged surface. The small local maximum for small values of x1 and x2 is barely visible.

However, as mentioned in Chapter III before, a line search method may have a chance of

getting stuck in it. For this reason, the simplex search technique was employed in the

following comparison of the different approaches to probabilistic optimization.


Figure 5.30: POS Surface Plot Over Design Space

In order to truly appreciate the advantages of POS as an objective function, the

following possible approaches to multi-objective probabilistic design/optimization are

compared to each other with the help of the example problem discussed above.

Considering the fact that F1 and F2 are explicitly dependent on Y1 and Y2, samples can be

generated based on the distributions for Y1 and Y2 and the optimal point for each sample

(y1, y2) can be determined using the MaxiMin, Overall Evaluation Criterion, or Goal

Attainment method. Once the optimal solution is found for each sample point, the

statistical mean of these solution points can be established and returned as the solution to

the multi-objective optimization problem. A visual representation of this approach can

be found in Figure 5.31 for 100 sample points.

Note that these methods cannot be considered accurate in yielding the optimal

solution, since the sample mean is based on the different solutions for the deterministic

design variables and not the function values for f1 and f2. Hence, they do not guarantee a


high value for the functions themselves or the probability of achieving a certain function

value level. In any case, as in Chapter II, the OEC seems to yield the more balanced

solution. In contrast, the solutions presented in Figure 5.32 are based on achievement

levels for the objectives f1 and f2 as elaborated next.

-2 -1 0 1 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Figure 5.31: Location of Optima ofMaxiMin, OEC, and Goal Attainment

Method for 100 Samples

-2 -1 0 1 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Figure 5.32: Location of Optima forDifferent Probabilistic Multi-Objective

Optimization Methods

Alternative approaches to obtaining a solution to the multi-objective probabilistic

optimization problem are the MaxiMin, OEC, or Goal Attainment method with objective

functions presented in Equations 3.5 and 3.6. Since both equations seek to maximize the

probability of achieving function values for F1 and F2 larger than 5, the actual objectives

(probabilities) are deterministic and all three methods can be employed in this multi-

objective optimization problem. No normalization is necessary, since the achievable

values for the objectives lie between 0 and 1. The goals for the Goal Attainment method

are naturally equal to 1, since it is the highest probability value attainable.

x2

x1

x2

x1

MaxiMinSolutions

OECSolutions

Goal AttainSolutions

Mean forOECMean for MaxiMin

And Goal Attain

JPDM/Simplex Search

JPDM/Line Search

P(F1 ≥ 5)

OEC

P(F2 ≥ 5)

Goal AttainMaxiMin


The solutions presented in Figure 5.32 indicate the differences in methods very

well. The solutions to the univariate optimization problems of maximizing the

probability of achieving F1 function values larger than 5 and maximizing the probability

of achieving F2 function values larger than 5 are added to the graph for reference

purposes. The two solutions generated by the Joint Probabilistic Decision Making

technique correspond to the two optimization techniques employed in conjunction with

POS, a sequential programming/line search and a simplex search technique.[Branch,

Grace, 1996]

The true test of superiority for all methods is listed in Table 5.1. All solutions

presented in Figure 5.32 produce a joint probability of achieving function values larger

than 5 as well as corresponding univariate probabilities. Clearly JPDM in conjunction

with the simplex search method yields the solution with the highest joint probability. The

JPDM/line search solution has a lower joint probability, but yields higher univariate

probabilities. This can be attributed to the fact that this solution produces a joint

probability distribution for F1 and F2 that allocates more probability density for higher

values of F1, as long as values for F2 stay small, and vice versa. The same phenomenon

can be observed for the solution produced by the Overall Evaluation Criterion method.

The solution produced by the single objective optimization problem of maximizing the

probability of achieving F1 function values larger than 5 marks the ideal solution for this

objective, which naturally cannot be achieved by any of the multi-criteria optimization

methods, due to the conflicting criteria. Nonetheless, its solution still yields a Probability

of Success value close to the OEC solution. This result cannot be generalized, which is


indicated by the single objective optimization problem of maximizing the probability of

achieving F2 function values larger than 5 which yields a joint probability of zero.

Finally, the low value in joint and univariate probability from the solutions found with

the MaxiMin and Goal Attainment method is due to the fact that both methods try to

achieve the same values for the univariate probabilities P(F1 ≥ 5) and P(F2 ≥ 5). It must

be concluded that these methods clearly are not suitable for solving probabilistic multi-

criteria optimization problems.

Table 5.1: Comparison of Joint and Univariate Probabilities

Solution Point(x1, x2)

JointP(F1 ≥ 5, F2 ≥ 5)

UnivariateP(F1 ≥ 5)

UnivariateP(F2 ≥ 5)

JPDM/Simplex Search (0, 1.5) 0.1794 0.4325 0.2449

JPDM/SQP/Line Search (0.0089, 1.0397) 0.1777 0.4658 0.2674

OEC (0.4608, 0.6606) 0.1565 0.5421 0.2678

MaxiMin (-0.178, 0.4234) 0.1101 0.2886 0.2882

Goal Attainment (-0.178, 0.4234) 0.1101 0.2886 0.2882

Single Obj. P(F1 ≥ 5) (0.7569, 0.7867) 0.1393 0.5552 0.2369

Single Obj. P(F2 ≥ 5)(0.026, -0.0832)

true: (0,0)0 0.0045 0.2949

Product Selection

While optimization is concerned with finding the best possible solution from a

very large pool of potential alternatives, product selection describes the process of

finding the best from a smaller pool of already “realized” alternatives. On the other hand,

product selection is often based on more criteria than optimization, as indicated in


Figure 5.2. A good example for the difference of product selection and optimization is

the difference between designing and buying a car. While the designer is looking for the

one best solution out of an almost infinite number of combinations of design parameter

values, the customer is looking for his best solution in a much smaller pool of alternatives

from different car manufacturers and models. For a more detailed description of the

differences between optimization and product selection see [Hwang, Masud, 1979],

[Hwang, Yoon, 1981], and [Sen, Yang, 1998].

As indicated in Chapter II, the main problem in multi-criteria product selection is

that of conflicting criteria, i.e. certain attribute values are approved upon with another

alternative only to the detriment of values for another attribute. The use of the Joint

Probabilistic Decision Making technique for product selection alleviates this problem by

transforming the multi-criteria problem into a single criterion problem: maximizing the

Probability of Success (POS). Each alternative in the product selection problem is

associated with a distinct joint probability distribution that yields a particular POS value

over the area of interest. The depiction in Figure 5.33 illustrates this concept by mapping

three joint probability distributions for three different alternatives over the area of

interest, which is spanned by two criteria. It is clear from the distribution’s overlap with

the area of interest, that Alternative #2 yields the highest POS, while Alternative #3

yields less and Alternative #1 the least amount of POS. So even though the criteria are

conflicting with each other, each alternative yields a unique value for POS that indicates

the usefulness to the decision-maker. It is this ability to rank the alternatives based on


their Probability of Success that is the true advantage of the Joint Probabilistic Decision

Making technique in product selection.

Criterion 1

Cri

teri

on 2

Alternative 1

Alternative 3Alternative 2

Area ofInterest

Figure 5.33: Comparing Joint Probability Distributions for Product Selection

To illustrate the use of the Joint Probabilistic Decision Making technique for

product selection, it is extended in Figure 5.34 to include the product selection process,

and an exit criterion. First the criteria are identified, based on which the best product is

selected. Second, probability distributions are assigned to the noise variables, i.e. the

variables that introduce the uncertainty into the product selection process. Third, control

variable values are established for each alternative. That is, each control variable can be

thought of as a vector, where each element in this vector corresponds to a different

(control variable) value for each alternative. Next, the joint probability distribution is

evaluated for each alternative through the use of one of the five schemes and the system

analysis tool. Finally, using the weights and criterion values that identify the area of

interest, the Probability of Success is calculated for each alternative.



1

CriterionValues

(zmin, zmax)


6Weights (w)7

4

Alternatives


YNon-Deterministic

ControlVariables

x = (0.5, 4, 0.07, …)

Deterministic


(Five Schemes)

POS

2

3

5

SatisfactoryPOS ?

Changez-Values?

ChangeWeights?


Y Y

YN

N N

Product Selection Process

8

91011

Figure 5.34: Joint Probabilistic Decision Making Technique for Product Selection

If at least one of the alternatives achieves a sufficient level of Probability of

Success in Element #9, the alternative with the highest value for POS is deemed the best

solution. If none of the alternatives reach levels of POS that seem reasonable, e.g. above

50%, the criterion values may be adjusted in Element #10 to yield higher values for POS.

On the same token, if several alternatives have a Probability of Success of one, the

criterion values should be adjusted to reduce the value of POS in order to be able to


distinguish the alternatives and determine which one is truly best. Note that by adjusting

the criterion values, i.e. changing the area of interest, the probabilistic analysis does not

need to be rerun, since the joint probability distributions for the different alternatives stay

constant. This sets the product selection process apart from the optimization loop, where

the joint probability distribution needs to be reevaluated for each optimization iteration,

while the criterion values stay constant. If changing the criterion values is not an option

or is not desired, and if the original set of weights does not reflect the criteria preference

structure of the decision-maker any longer, preference weights can be adjusted in

Element #11.

Change of Preferences

If the decision-maker decides to look at a different preference structure among the

criteria, new weight values need to be assigned to all criteria. These new weights yield a

change in target values, i.e. changing the area of interest. The modified area in return

yields a different POS value for each alternative and possibly a different ‘best’ solution.

This process can be repeated until the decision-maker has a sufficient understanding of

the dependence of the best design alternatives on the distribution of weights.

Weight Adjusted Target Values

The technique suggested in this thesis accounts for preferences among criteria by

weight adjusted target values t that limit the area of interest:

( ) minmin zwt ⋅⋅= N , (5.31)


and ( )N⋅=

wz

t maxmax (5.32)

with N being the number of criteria. This formulation essentially narrows the target

range of interest for the criteria with high preference weights and widens its range of

interest for the ones with low weights. The formulation for the Probability of Success in

Equations 5.26 and 5.27 consequently changes to:

∑=

≤≤=M

jjjj tztI

MPOS

1maxmin )(

1(5.33)

for the Empirical Distribution Function (Equation 5.3), with M being the number of

samples, and ∫=max

min

)(t

t

ss dfPOS (5.34)

for the Joint Probability Model (Equation 5.12). It is essential to note that the POS in this

case does not represent the numerical probability of achieving values within the area of

interest spanned by the criteria, but rather yields a measure of ‘goodness’ for the design

alternatives, accounting for preferences among decision criteria. Hence, products fielded

based on this formulation promise to yield a higher customer satisfaction than products

that were fielded based on the aforementioned Probability of Success without criteria

preferencing.15 This concept is also visualized in Figure 5.35, where one design solution

(Alternative 1) appears to be superior when no preferencing is applied, while the other

solution (Alternative 2) appears to be the better one when preferencing is applied.

15 Preferencing = Assigning weighting values to criteria in a decision making environment.


z2

Area ofInterest

Alternative 1

Alternative 2

Without Preferencing

z1z1min z1max

z2min

z2max

z2

Area ofInterest

Alternative 1

Alternative 2

With Preferencing

z1t1min t1max

t2min

t2max

Figure 5.35: Comparison of Alternatives Based on POS with or without Preferencing

Requirement Trade-Off

While analysis and drafting are important activities in design, no design process is

complete without a needs analysis that includes a gathering and investigation of customer

requirements. For complex products or markets, the designer is often faced with the

dilemma of multiple conflicting requirements. These conflicts have always been

explored, if not solved, through the use of requirement trade-off techniques.[Dieter,

1991] Siddall, for example, introduced an approach of Interaction Curves,[Siddall, 1982]

which represent the locus of all ‘optimal designs’ in a Pareto sense, where “each point on

the line is a trade-off point.”[Siddall, 1982] Other names for this curve, such as Pareto-

Front or Pareto-Line, are also common in the literature.[Hwang, Masud, 1979], [Hwang,

Yoon, 1981], [Zeleny, 1982], [Osyczka, 1984], [Steuer, 1986], [Stadler, Dauer, 1992]

Whether through a formal approach or a mind experiment, designers have always used

the information from Interaction Curves, or the like, to answer such questions as: “What

happens to Requirement B, if I was able to relax Requirement A a little?” “How much do


I have to relax Requirement A in order to meet my Requirement B?” and so on. It is

these types of questions that are addressed through the use of the Joint Probabilistic

Decision Making technique for requirement trade-offs.

In principal, the technique computes the new Probability of Success for each

combination of requirement values, thereby identifying the impact of criterion target

value changes on the alternatives. To visualize this process, a plot with lines of constant

POS is created, similar to the ones in Figure 5.36 and 5.37. Each point (z1, z2) in this plot

represents the joint cumulative probability of achieving values smaller (or larger) than z1

for the one, and smaller (or larger) than z2 for the other requirement. If the area of

interest is not open ended, i.e. zimin Å -∞, or zimax Å ∞, both limits have to be traded

separately. In general two requirement trade-off scenarios are possible. First, one or

more requirements can be relaxed in order to gain some Probability of Success. This

scenario is exemplified in Figure 5.36, where the requirement Z2 > t2min is relaxed to

Z2 > t2*

min, thereby increasing the joint probability from 0.22 to 0.5. In the second

scenario, POS is kept constant, while one requirement is relaxed to allow for more

stringent values of another requirement. In other words, values for one requirement are

being ‘traded in’ for values of another. This scenario is demonstrated in Figure 5.37,

relaxing the requirement Z2 > t2min to Z2 > t2*min in order to allow the requirement

Z1 > t1min to be tightened to Z1 > t1*min, while keeping the Probability of Success constant.


IncreasedProbability

∞=max2t

∞=max1t

t2 min

t2*

min

Lines of ConstantCumulative Probability

z1

z2

0.01

0.10.2

0.8

0.9

t1 min

0.99

rela

x

0.5

0.22

Figure 5.36: Requirement Trade-Off toGain POS

Lines of ConstantCumulative Probability

z1

z2

0.01

0.10.2

0.80.9

t1 min

0.99

t2 min

t2*

min

t1*

min

rela

x

tighten

∞=max2t

∞=max1t

Figure 5.37: Trade-Off to TightenRequirement

To illustrate the use of the Joint Probabilistic Decision Making technique for

product selection, it is extended in Figure 5.38 to include the requirement trade-off

process, and an exit criterion. First the criteria are identified, which provide the

requirements to be traded. Second, probability distributions are assigned to the noise

variables, i.e. the variables that introduce the uncertainty to the systems design process.

Third, control variable values are established for each alternative, similar to the product

selection process. Next, the joint probability distribution is evaluated for each alternative

through the use of one of the five schemes and the system analysis tool. Finally, using

the weights and criterion values that identify the area of interest, the Probability of

Success is calculated for each alternative. If the requirements need to change in

Element #9, the criterion values are adjusted in Element #10 and the new Probability of

Success is computed. This process is repeated until the decision-maker has gained

sufficient insight into how the solution depends on changes in the requirements or until a

sufficient level of POS is reached for the best solution. Note that the probabilistic


analysis does not need to be rerun for the requirement trade-off process, since only the

area of interest changes while the joint probability distributions for the different

alternatives stay constant.


1

CriterionValues

(zmin, zmax)


6Weights (w)7

4

Alternatives


YNon-Deterministic

ControlVariables

x = (0.5, 4, 0.07, …)

Deterministic


(Five Schemes)

POS

2

3

5

Change inRequirements?


Y

NRequirement

Trade-Off Process

Changez-Values

8

910

Figure 5.38: Joint Probabilistic Decision Making Technique for Requirement Trade-Offs

To highlight the significance of the Joint Probabilistic Decision Making technique

to aerospace systems design, the following chapter employs several example applications

for optimization, product selection, and requirement trade-offs.

Oliver Bandte Chapter VI 122

CHAPTER VI

IMPLEMENTATION INTO SYSTEMS DESIGN

Feasibility Problem

The first application example demonstrates the use of the Joint Probabilistic

Decision Making technique for the “system reliability” or “determination of feasible

space” problem. This problem is not a decision making problem in the strictest sense,

however, it facilitates the design process, particularly the technology selection

process.[Mavris, Kirby, 1998],[Mavris, Kirby, Qiu, 1998] The aim is to determine as

early as possible in the design phases whether a particular design concept has a chance of

having a feasible design solution.16 A feasible design is defined as a design that satisfies

all imposed constraints.[Reklaitis, Ravindran, Ragsdell, 1983] Typical constraints in

aircraft design are limitations on approach speed, landing and take-off field length, and

aircraft noise based on FAA regulations. The sketch in Figure 6.1 illustrates this notion

for a simple example with two design variables and two constraints. The whole rectangle

denotes the design space, i.e. all possible design variable setting combinations. The dark

lines mark the constraints as functions of the design variables for a particular constraint

value that needs to be satisfied. The white area in the middle denotes the feasible space.

16 The main difference between a design solution and design concept lies in the level of determination ofthe design variables. It is a solution, if all variable settings are known. It is a concept, if only few settingsare determined, but the products major functionalities and components are known.[Dieter, 1991]


Constraint 2

Constraint 1

Design Variable 1

Des

ign

Var

iabl

e 2

Figure 6.1: Feasible Space in a Design Space

The computation of a deterministically formulated feasibility requires an

exorbitant amount of work, which can be reduced through the use of probability theory.

Such a formulation assumes uniform distributions for the design variables, i.e. assigning

each possible variable value the same probability of occurrence. The feasibility problem

is thereby reduced to determining the probability of satisfying all constraints

concurrently. This probability is equivalent to the volume spanned over the feasible area

by the joint uniform distribution and is therefore a measure of feasibility. With the

previously described probabilistic process, and recognizing that the constraints are simply

random variables, since they are a function of the design variables, a joint probabilistic

approach can be used to determine system feasibility.

This technique is very useful in conceptual and preliminary design, where concept

feasibility needs to be evaluated quickly in order to determine whether a particular

concept should enter the next design phase. For example, if little feasibility is found, new

technologies may be introduced to the design concept in hope of increasing the feasible

space. Through repeated execution of this method, different technologies can be applied

to the system, while a growth or shrinkage of feasible space manifests their benefit. This

process has been formulated in [Mavris, DeLaurentis, 1998] as a five step approach to

aircraft design, suggesting the use of the Fast Probability Integration tool (FPI)


[Southwest Research Institute, 1999] for determining the feasibility. From the flow chart

in Figure 6.2 one can see that the determination of feasibility is the key step in this

process, evaluating whether an expensive investigation of new technologies is necessary.

YN

N

P(feas) < εsmall

Problem DefinitionIdentify objectives, constraints,

design variables (and associated side constraints), analyses,

uncertainty models, and metrics

1

x 1

x 2

x 3

Determine System Feasibility

Des

ign

Spa

ce M

odel

Constraint Fault Tree

C1 C2 C3 C4

AND

P(feas)

2

FPI(AIS) or Monte Carlo

Relax Constraints? Y

Examine Feasible Space

x1

x2

x3

3Constraint

Cumulative Distribution Functions (CDFs)

C1

P

C2

P

C3

P

Des

ign

Spa

ce M

odel

FPI(AMV) or

Monte Carlo

Relax Active Constraints

?

Y

Technology Identification/Evaluation/Selection (TIES)

Ci

P

Old Tech. New Tech.

Obtain New CDFs• Identify Technology Alternatives

• Collect Technology Attributes

• Form Metamodels for Attribute Metrics

through Modeling & Simulation

• Incorporate Tech. Confidence Shape Fcns.

• Probabilistic Analysis to obtain CDFs for the

Alternatives

4

5Decision Making• MADM Techniques

• Robust Design Simulation • Incorporate Uncertainty Models

• Technology Selection • Resource Allocation • Robust Design Solution

Figure 6.2: Five Steps to Aircraft Design [Mavris, DeLaurentis, 1998]

The use of the Joint Probabilistic Decision Making technique for determining

feasibility replaces the “AND” in Step #2 of the flow chart in Figure 6.2, calculating the

Probability of Success rather than P(feas). As an additional benefit from the use of the

JPDM technique, Step #3 is automatically executed when calculating POS, i.e. it does not


need to reapply the Uniform-Distributions to find the Constraint Cumulative

Distributions, as required by FPI. A reformulated flow chart that includes the Joint

Probabilistic Decision Making technique can be found in Figure 6.3. The other

significant advantage of JPDM over FPI is the fact that FPI’s system feasibility analysis

does not yield values smaller than 0.5, which can only be determined after all of the

computational effort has been expended.

Problem Definition

Identify objectives, constraints, design variables (and associated

side constraints), analyses,uncertainty models, and metrics

Problem Definition

Identify objectives, constraints, design variables (and associated

side constraints), analyses,uncertainty models, and metrics

1

Determine System Feasibility

x1

x2

x3

Des

ign

Spa

ce M

odel

System AnalysisC1 = f(x1, x2, x3)C2 = f(x1, x2, x3)C3 = f(x1, x2, x3)

JPDM

PC1 = P(C1< Target1)



P(feas) = P(C1< Target1 ∧ C2< Target2 ∧ C3< Target3)

2

Decision Making

•JPDM•Robust Design Simulation

•Incorporate Uncertainty Models

Decision Making

•JPDM•Robust Design Simulation

•Incorporate Uncertainty Models

5

3

P(feas)< εsmall

P(feas)< εsmall

Relax ActiveConstraints ?

Relax ActiveConstraints ?

YYN

•Technology Selection•Resource Allocation•Robust Design Solution

N

Technology Identification/Evaluation/Selection (TIES)

• Identify Technology Alternatives• Collect Technology Attributes• Form Metamodels for Attribute Metrics

through Modeling & Simulation• Incorporate Tech. Confidence Shape Fcns.• Prob. Analysis to Obtain CDFs for Alternatives

4

Obtain New CDFs

Old Tech.New Tech.

Ci

P

Figure 6.3: Five Steps to Aircraft Design with JPDM

A particularly good example for the determination of feasibility is the supersonic

transport aircraft, depicted in Figure 6.4. This next generation aircraft has very little

chance of satisfying all imposed constraints with today’s technology, due to its stringent


performance requirements. In other words, this concept at today’s technology levels has

a very small feasible space, if any at all. The baseline aircraft has an area-ruled fuselage

(maximum diameter of 12 ft.), a double delta planform, and four nacelles below the wing,

housing mixed flow turbofan power plants. The values for some of the important design

parameters are given in Table 6.1. Due to the severe impact of the sonic boom on the

ground, it was decided that the aircraft is not allowed to fly supersonically over largely

populated areas. Since many cities are not located adjacent to a large body of water, a

split supersonic/subsonic mission is assumed to be required. Unfortunately, the

requirement of subsonic cruise penalizes the design, since the aerodynamic shape can not

be optimized for pure supersonic flight, yielding a compromised design with higher drag

during supersonic cruise. As a consequence, the aircraft needs to carry more fuel, which

in turn increases the gross weight. The length of the subsonic cruise segment is assumed

to be 15% of the design range, which equals 750 nm.

Figure 6.4: Supersonic Transport Concept

Table 6.1: Description of the Baseline

Parameter BaselineRange 5000 nmPayload 300 PassengersFuselage length 310 ft.Span 77.5 ft.Inboard Sweep 74 deg.Outboard Sweep 45 deg.Mach Number 2.4Supersonic Cruise Altitude ~63,000 ft.

As demonstrated by the subsonic flight requirement over land, designing a

supersonic transport vehicle is a multidisciplinary and difficult task. Choosing a wing

planform shape, for example, is driven by the need for efficient performance at both sub-


and supersonic cruise conditions, a conflicting design objective in itself.[DeLaurentis,

Mavris, Schrage, 1996], [Sakata, Davis, 1977] Furthermore, the trades involved in

planform selection are complicated by different discipline considerations for

aerodynamics, structures, propulsion, etc., and the presence of design and performance

constraints at the system level which are directly related to the wing. The limit on

approach speed, for example, is the main driver for the wing loading. On the other hand,

fuel volume requirements impact the wing size and shape. Both become sizing criteria

that, treated as constraints, tend to increase the wing in size. On the other hand, increased

wing area yields higher induced and skin friction drag, thus increasing fuel consumption.

These examples are just a snap shot of design trades that drive the design process and

give rise to performance constraints, which are complemented by such Federal Aviation

Regulations as the takeoff and landing field length limitations (less than 10,500 ft).

Since the feasibility study is just one application for the joint probability

formulation introduced in this thesis, only a simple example for two constraints, approach

speed and take-off field length, is executed here. Specifically, noise constraints are not

considered, since they cannot be satisfied with the simulated, current technology. The

design variables used in this feasibility study are listed in Table 6.2, representing some of

the key drivers in aircraft design.[Mavris, Bandte, Schrage, 1996] They have been

selected from an elaborate screening process that identified the most significant

variables.[DeLaurentis, Mavris, Schrage, 1996] An illustration of the kink location in a

notional wing planform can be found in Figure 6.5. Finally, as indicated in Figure 6.3,

uniform distributions are assigned to all variables based on their ranges in Table 6.2.


Table 6.2: Design Variable Description and Range

Variable Name RangeThrust to Weight Ratio TWR 0.28 - 0.32Wing Area WingArea 8.5 - 9.5 * 103 ft2

Longitudinal Kink Location x1 154% - 162% SemispanSpanwise Kink Location y1 50% - 58% SemispanTurbine Inlet Temperature TIT 3 - 3.25 * 103 °FFan Pressure Ratio FPR 3.5 - 4.5

x1

y1

Figure 6.5: Illustrationof the Kink Location

To determine the feasibility, the eight step Joint Probabilistic Decision Making

technique is executed as outlined in Figure 6.6. The first step identifies the criteria: the

two constraints take-off field length (TOFL) and approach speed (Vapp). In Step #2,

uniform distributions are assigned to the variables over their respective ranges, identified

in Table 6.2. In the third step then, values are assigned to the remaining (control)

variables that describe the concept. The system analysis tool is identified in Step #4 to be

the aircraft synthesis/sizing code, Flight Optimization System (FLOPS),[McCullers,

1994] evaluating TOFL and Vapp based on the noise and control variables. To determine

the joint probability distribution, Scheme #I was chosen in Step #5, generating 10,000

Monte-Carlo simulation samples with the system analysis tool for the Empirical

Distribution Function, Equation 5.4, of TOFL and Vapp. The samples are collected in

pairs and are thus distributed jointly. In Step #6, minimum and maximum values for the

criteria are established that limit the area of interest: TOFLmin = 0 ft (physical limit),

TOFLmax = 10,500 ft (FAA Regulation), Vappmin = 0 kts (physical limit), and

Vappmax = 154 kts (FAA Regulation). Finally, Step #8 combines the area of interest with

the joint probability distribution and determines the Probability of Success, i.e.

feasibility. Note that Step #7, assigning weights to the criteria, has been omitted here,


since it does not seem sensible to have different preferences for the different constraints.

Explicitly, the feasibility is calculated to be:

.0107.0)154 ,500,10(

)154 ,500,10()154 ,500,10(

000,10

1000,101 =≤≤

==≤≤

∑=

ktsaftaI

ktsftFktsVAPPftTOFLP

iVAPPi

iTOFL

CriteriaTOFL and Vapp

1

Analysis ToolFLOPS

Criterion ValuesTOFLmin= 0 ftVappmin= 0 kts

TOFLmax= 10500 ftVappmax= 154 kts

6

4

Baseline

Uncertainty(Design) Variables

Table 6.2Non-Deterministic

ControlVariables

Concept/Table 6.1Deterministic

Joint ProbabilityDistribution:Schemes #I

POS =Feasibility

2

3

5

8

Figure 6.6: Determination of Feasibility with JPDM

In other words, the chance of finding a feasible design for the Supersonic

Transport within the specified design space, using current technologies, is 1.07%. Only

the infusion of new technologies can increase the design’s feasibility. For a visual

representation of the EDF, its graph is depicted in Figures 6.7 and 6.8 in two and three

dimensions respectively. The graph in Figure 6.7 also displays the area of interest, i.e.

area of feasible solutions, limited by the values for TOFLmax and Vappmax.


0.85 0.9 0.95 1 1.05 1.1 1.15 1.2

x 104

155

160

165

170

175

180

TOFL

Vap

p

Figure 6.7: Joint EDF for Take-OffField Length and Approach Speed (2D)

80008600

92009800 10400 11000

1160012200

Figure 6.8: Joint EDF for Take-OffField Length and Approach Speed (3D)

Since the display of the EDF in Figure 6.7 is not particularly aesthetic,

Scheme #II is employed to produce graphs for the Joint Probability Model, displayed in

Figures 6.9 and 6.10 for two and three dimensions respectively. The regression of the

data yields a Normal-Distribution with N(9691.3, 510.55) for the take-off field length and

a Normal-Distribution with N(160.3, 3.0) for approach speed. The correlation coefficient

is estimated to be 0.7311.

0.9 1 1.1 1.2

x 104

155

160

165

170

175

180

185

TOFL

Vap

p

Figure 6.9: Joint Normal Distribution forTake-Off Field Length and Approach

Speed (2D)

Figure 6.10: Joint Normal Distribution forTake-Off Field Length and Approach

Speed (3D)

Area ofInterest

Area ofInterest

(kts

)

(ft) (ft)(kts)

(ft)

(kts)

(kts

)

(ft)


Optimization of a Supersonic Commercial Transport

In order to demonstrate the use of the Joint Probabilistic Decision Making

technique for optimization in aerospace systems design, the following problem is posed:

An aircraft manufacturer plans to build a new supersonic passenger transport aircraft that

is supposed to be operated by all major airlines around the world. Initial studies indicated

that the aircraft should fly at a cruise speed of Mach 2.4, has an area-ruled fuselage

(maximum diameter of 12 ft.), a double delta planform, and four nacelles below the wing,

housing mixed flow turbofan power plants, similar to the concept shown in Figure 6.11.

However, the final design of the aircraft is not determined and subject to an optimization

problem, maximizing the chance for the manufacturer and the airline to make a profit.

Figure 6.11: Notional Supersonic Transport

Specifically, the optimal values for the number of total passenger capacity,

number of first class passengers, maximum (design) range, thrust-to-weight ratio, and

wing area are to be determined. They need to maximize the probability of the Supersonic

Transport achieving return on investment values larger than 10% for the airline and 12%

for the manufacturer. However, in order to limit the scope of the design problem, limits,

outlined in Table 6.3, are imposed on these design variables.


Table 6.3: Range of Allowable Design Variable Values for Optimization

Thrust-Weight Wing Area Max. Range # of First ClassPassengers

Total # ofPassengers

Minimum 0.28 8500 ft2 5000 nm 6 250

Maximum 0.32 9500 ft2 6500 nm 60 350

Special attention needs to be directed to the anticipated mission the supersonic

transport is intended to fly, i.e. cities it can serve. Due to the severe impact of the sonic

boom on the ground, it was the decided that the aircraft is not allowed to fly

supersonically over largely populated areas. Since many cities are not located adjacent to

a large body of water, a split supersonic/subsonic mission is assumed to be required.

Unfortunately, the requirement of subsonic cruise penalizes the design, since the

aerodynamic shape cannot be optimized for pure supersonic flight, yielding a

compromised design with higher drag during cruise. As a consequence, the aircraft needs

to carry more fuel, which in turn increases the gross weight. On the other hand, to

require a section of subsonic cruise for service to cities that are 6500 nm apart would

penalize the design too much, since the only cities that far apart and serviced by a direct

flight, are Los Angeles and Sydney, both on the Pacific Ocean.[Oxford, 1999] Hence, it

is assumed that an aircraft designed for a maximum range of 6500 nm with no subsonic

cruise has enough room for fuel to fly a mission of 5000 nm which includes up to

1500 nm of subsonic cruise. A linear relationship is developed and employed for the

sizing routine to calculate a feasible subsonic cruise length, given a particular maximum

range. This relationship, Equation 6.1, corresponds particularly well to actual distances

of major world cities and respective lengths of flight over land.[Oxford, 1999]


Subsonic Cruise Distance = 6500 nm – Maximum Range (6.1)

Important to note is that the aircraft has to satisfy certain physical and operational

requirements as well as constraints imposed by the Federal Aviation Agency. Namely,

the landing speed has to be less than 154 knots, landing and take-off field lengths need to

be less than 10,500 ft, and the take-off gross weight has to be less than 1 million pounds

in order for the aircraft to land on existing runways.

Since the objective of this optimization problem is to maximize the probability of

achieving return on investments larger than 12% and 10% for the manufacturer and

airline concurrently, special attention needs to be paid to its economic analysis.

Unfortunately, the manufacturer has little to no control over the key economic parameters

and optimizing them is impossible. The approach taken here, as outlined in Chapters III

and V, is to assign probability distributions to the key economic parameters of interest,

which are summarized in Table 6.4. They have been selected from an elaborate

screening process that identified the most significant economic variables.[Mavris,

Bandte, Schrage, 1995]

The following economic parameters affect the airline only: Load Factor, which is

the ratio of the equivalent full fare booked seats to the number of available seats; Average

Yield, which represents the airline’s average yield per revenue passenger mile; Fuel

Price; Economic Range, which is the average distance between city pairs the aircraft is

scheduled to connect; Utilization; and %Subsonic Cruise, which is the average

percentage of the Economic Range flown in subsonic cruise. %Subsonic Cruise itself

depends on the Economic Range and decreases with increasing Economic Range values


linearly. For example, if the Economic Range is 6500 nm, %Subsonic Cruise is zero; if

the Economic Range is 3000 nm, %Subsonic Cruise is 50%, based on the argument made

earlier. The linear relationship is again in correspondence with actual distances of major

world cities and respective lengths of flight over land.[Oxford, 1999]

Table 6.4: Economic Parameter Distributions

Economic Parameter Range Unit DistributionType

Parameters

Load Factor 65 - 85 % Beta α = 3, β = 3

Average Yield (Coach)

Average Yield (First)

0.09 – 0.15

0.13 – 0.19$ Beta α = 5, β = 3

Fuel Price 0.6 - 0.9 $/gal Beta α = 3, β = 5

Economic Range 3000 – max Range nm Beta α = 2, β = 4

Utilization 4500 - 5500 hrs/yr Beta α = 3, β = 3

% Subsonic Cruise 0 - 50 % Beta α = 3, β = 3

Inflation 1 – 9 % Beta α = 3, β = 5

Aircraft Price

250 - 350

300 - 400

350 - 450

M$ Beta α = 3, β = 3

Learning Curve 0.75 - 0.85 Beta α = 3, β = 3

Production Quantity 400 - 800 Beta α = 3, β = 3

Engineering Labor Rate 50 - 100 $/hr Beta α = 3, β = 5

Tooling Labor Rate 40 - 90 $/hr Beta α = 3, β = 5

Years Until Production 3 - 5 yrs DiscreteP(3yrs) = 1/3P(4yrs) = 1/3P(5yrs) = 1/3

The manufacturer, on the other hand, is affected by the following variables:

Learning Curve, which represents the reduction in unit production cost with increasing

numbers of aircraft produced; Production Quantity; Engineering Labor Rate; Tooling

Labor Rate, which represents the labor rate for all production work; and Years Until


Production, which entails most of the detailed engineering work. Inflation and Aircraft

Price affect the airline as well as the manufacturer. In particular, Aircraft Price is

assumed to be dominated by the market, therefore uncertain, but correlated with the

Maximum Range and Total Number of Passengers. Hence, aircraft with a Maximum

Range of less than 5750 nm and less than 300 passengers are assumed in this example to

reach price levels between $200 and $300 million, aircraft with a Maximum Range of

less than 5750 nm or less than 300 passengers are assumed to reach price levels between

$250 and $350 million, and aircraft with a Maximum Range larger than 5750 nm and

more than 300 passengers are assumed to reach price levels between $300 and $400

million.

To determine the probability of satisfying both the manufacturer and the airline in

their demands for return on investment, the ten step Joint Probability Decision Making

technique for optimization is executed as outlined in Figure 6.12. The first step identifies

the criteria to be the two return on investments for the airline (ROIA) and manufacturer

(ROIM). In Step #2, the distributions are assigned to the variables over their respective

ranges identified in Table 6.4. In the third step then, values are assigned to the remaining

(control) variables that describe the concept. The system analysis tool, identified in

Step #4, is a combination of the aircraft synthesis/sizing code FLOPS [McCullers, 1994]

and the Aircraft Life Cycle Cost Analysis program ALCCA [Galloway, Mavris, 1993].

To determine the joint probability distribution, Scheme #I was chosen in Step #5,

generating 1,000 samples with the system analysis tool for the Empirical Distribution


Function of ROIM and ROIA, based on a Monte-Carlo simulation. The samples are

collected in pairs and are thus distributed jointly.

Analysis ToolFLOPS/ALCCA

WeightswROIM = 0.5wROIA = 0.5

7

4

Baseline



ControlVariables

Table 6.3Deterministic

Joint ProbabilityDistributionScheme #I

POS

2

3

5

UpdateDeterministicVariables inOptimization

Scheme

SatisfactoryPOS ?


Y

N

CriteriaROIM and ROIA

1

9

Optimization Loop

8

10

Criterion ValuesROIMmin = 12%

ROIMmax = ∞ROIAmin = 10%

ROIAmax = ∞

6

Figure 6.12: JPDM as Optimization Process

In Step #6, minimum and maximum values for the criteria are established that

limit the area of interest: ROIMmin = 12 %, ROIMmax = ∞, ROIAmin = 10 %, and

ROIAmax = ∞. In Step #7 equal weights of 1/N = 0.5 are assigned to both criteria, since


no preference of one criterion over the other is identified for the optimization. Finally,

Step #8 combines the area of interest with the joint probability distribution and

determines the Probability of Success, i.e. the chance of satisfying both return on

investment requirements concurrently. As long as the achieved POS can still be

improved upon, the optimization loop continues to update the deterministic design

variables. The optimization routine used for this research is the line search technique for

constrained optimization, which is part of the optimization package in

MATLAB®.[Branch, Grace, 1996] After 71 iterations, the final setting for the control

variables is determined to be Thrust-to-Weight ratio = 0.312021, Wing Area = 8576 ft2,

Maximum Range = 5000 nm, Number of First Class Passengers = 14, and Total Number

of Passengers = 350, yielding a Probability of Success of 0.782. The iteration history is

displayed in Figure 6.13. The constraint on approach speed (Vapp) is the most stringent

for this example and has been included in the optimization history of Figure 6.13. Note

that the optimizer first tries to find an area of large objective function values and then

reduces the values for Vapp.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 11 21 31 41 51 61 71

Iteration

PO

S

148

150

152

154

156

158

160

162

Vap

p (

kts) POS

Vapp

Constraint

Figure 6.13: Optimization Iteration History of POS and Vapp


On the other hand when using an Overall Evaluation Criterion as an objective

function, %)10(%)12(OEC 21

21 ≥+≥= ROIAPROIMP , the best solution is found at

Thrust-to-Weight Ratio = 0.310435, Wing Area = 8676 ft2, Maximum Range = 5000 nm,

Number of First Class Passengers = 59, and Total Number of Passengers = 346, yielding

an OEC value of 0.8845. The POS value of 0.773 for this OEC solution is smaller than

the previous solution based on the Probability of Success. Also, finding the OEC

solution took 109 iterations, as indicated in the optimization history of Figure 6.14, while

the POS solution needed only 71. In conclusion, the OEC is a less adequate objective

function for this aerospace systems optimization problem than POS.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106

Iteration

PO

S

OE

C

148

153

158

163

168

Vap

p (k

ts) OEC

POS

Vapp

Constraint

Figure 6.14: Optimization Iteration History of OEC, POS, and Vapp

Comments and Recommendations

While the computer code FLOPS constitutes the best publicly available routine

for synthesis and sizing of aircraft, it imposes a limitation on the sizing of the fuselage

that should be commented on. Fuselage diameter and length are as much inputs to

FLOPS as the number of passengers, i.e. all three are modeled as independent variables.


This is clearly not true for sizing of a real aircraft: fuselage length or diameter will

increase with the number of passengers. Hence, for the demonstrated optimization

exercise the fuselage length and diameter was kept constant, thereby underpredicting the

fuselage skin friction and wave drag of the larger vehicles. As a consequence, their

operating cost is a little higher than predicted by ALCCA, producing a lower return on

investment for the airline. However, this overprediction of ROIA affects POS as much as

the OEC and the conclusion remains the same: POS is the more adequate objective

function for aerospace systems design, yielding a higher Probability of Success in less

iterations for this example.

A last comment should be made about the line search optimization routine used

for this exercise. While this routine is the simplest technique for optimization, it

highlights the benefits of the use of POS over the OEC by requiring fewer iterations with

POS as an objective function. By using more sophisticated optimization routines the total

number of iterations for both objective functions should be decreased, with the potential

for a less pronounced advantage of POS over the OEC in terms of iterations. However,

there is no evidence that a different optimization routine will enable the OEC to find a

better solution. On the contrary, the better the optimization routine, the better the

chances that it finds the true optimum in Probability of Success with POS as an objective

function rather than the OEC. This is due to the fact that customer satisfaction is

described by a joint probability (POS) rather than a summation of independent univariate

probabilities (OEC). However, an investigation into different optimization routines has

not been part of this research and is left for future studies.


Product Selection

The third application example of the Joint Probabilistic Decision Making

technique is a product selection problem: A large international airline intends to extend

their current fleet by purchasing a new system. The new aircraft are supposed to be used

on long haul routes with high passenger demand. The systems considered are the Boeing

B-747-400, Boeing B-777-200, Airbus A340-300, and a notional Supersonic Transport.

The Supersonic Transport seems exotic next to the three subsonic aircraft, but is of

particular interest to an airline. Due to its increased flight speed, the Supersonic

Transport is able to sustain travel times cut down to half of what a subsonic aircraft is

able to offer. This makes the Supersonic Transport, and consequently the airline that

operates it, very attractive for business travelers and people with very short vacation

periods. Both groups are the airlines’ best paying customers, having sustained their

business for years when air travel was down significantly. It is therefore anticipated, that

a Supersonic Transport would be received very well by the airlines’ customers. A visual

representation of each system can be found in Figures 6.15 to 6.18 and the characteristics

for each system are listed in Table 6.5, where Fuel Weight measures the amount of fuel

used when traveling the full range at full capacity.

Figure 6.15: B-747 [Boeing, 2000] Figure 6.16: B-777 [Boeing, 2000]


Figure 6.17: A340 [Airbus, 2000] Figure 6.18: Supersonic Transport

Table 6.5: Four Alternatives for Product Selection Problem

Characteristic Boeing B-747 Boeing B-777 Airbus A340 SupersonicTransport

Aircraft Price M$150 - 200 M$110 - 160 M$100 - 150 M$200 - 300

Total # of Seats 412 375 330 342

# First Class Seats 82 20 32 60

Range 7500 nm 4600 nm 6600 nm 5000 nm

Cruise Speed Mach 0.85 Mach 0.84 Mach 0.8 Mach 2.4

Gross Weight 868013 lbs 535000 lbs 558900 lbs 882807 lbs

Fuel Weight 343710 lbs 156735 lbs 224733 lbs 532946 lbs

# of Engines 4 2 4 4

Following the ten-step process, outlined in Figure 6.19, the criteria based on

which the aircraft system is selected are identified first. For this example, return on

investment (ROI), total operating cost in million dollars per year (TOC) averaged over the

economic life of the aircraft (20 years), and annual revenue in million dollars (REV) are

selected as the three criteria. As a second step, probability distributions are assigned to

the uncontrollable variables, which have been selected from an elaborate screening

process that identified the most significant economic variables.[Mavris, Bandte, Schrage,

1995] These variables are listed in Table 6.6 with their respective distributions that

represent the likelihood of those values occurring during the 20 years of operation.


CriteriaROI, REV, and TOC

1

Criterion ValuesROImin = 15%

ROImax = ∞REVmin = M$100

REVmax = ∞TOCmin = M$0

TOCmax = M$90

Analysis ToolALCCA

6WeightswROI = 1/3wREV = 1/3wTOC = 1/3

7

4

Alternatives



ControlVariables

Table 6.5Deterministic

Joint ProbabilityDistributionScheme #I

POS

2

3

5

SatisfactoryPOS ?

Changez-Values?

ChangeWeights?


Y Y

YN

N N

Product Selection Process

8

91011

Figure 6.19: JPDM for Product Selection

The Load Factor is the ratio of the equivalent full fare booked seats to the number

of available seats. Due to the decreased travel time, the airline is assumed to be able to

charge a three cent premium for the notional Supersonic Transport in the coach and first

class sections. However, Table 6.6 only lists the Average Yield, representing the

airline’s average yield per revenue passenger mile, for the subsonic aircraft. The


Economic Range depicts the average distance between city pairs the aircraft is scheduled

to connect, while the Inflation is the average anticipated inflation rate over the economic

life. The % Subsonic Cruise, which is the average percentage of the Economic Range

flown in subsonic cruise, is obviously only used for the Supersonic Transport. It depends

on the Economic Range and decreases for increasing Economic Range values linearly.

For example, if the Economic Range is 6500 nm, % Subsonic Cruise is zero; if the

Economic Range is 3000 nm, % Subsonic Cruise is 50%. The linear relationship is again

in correspondence with actual distances of major world cities and respective lengths of

flight over land.[Oxford, 1999] The price of the aircraft system obviously varies from

alternative to alternative (see Table 6.5), but the same type of distribution is used in the

analysis of all competing systems. Passenger and Baggage Weight represent average

values for the all passengers and bags.

Table 6.6: Noise Variable Distributions

Noise Variable Range Unit DistributionType

Parameters

Load Factor 65 - 85 % Beta α = 3, β = 3

Average Yield (Coach)

Average Yield (First)

0.06 – 0.12

0.10 – 0.16$ Beta α = 5, β = 3

Fuel Price 0.6 - 0.9 $/gal Beta α = 3, β = 5

Economic Range 3000 – max Range nm Beta α = 2, β = 4

Utilization 4500 - 5500 hrs/yr Beta α = 3, β = 3

% Subsonic Cruise 0 - 50 % Beta α = 3, β = 3

Inflation 1 – 9 % Beta α = 3, β = 5

Aircraft Price See Table 6.5 M$ Beta α = 3, β = 3

Passenger Weight 150 - 210 lbs Normal µ = 180, σ = 10

Baggage Weight 29 - 59 lbs Normal µ = 44, σ = 5


The third step is comprised of identifying and assigning values to the control

variables, i.e. parameters that distinguish the different alternatives from each other.

Table 6.5 provides these values to a large extent. More detailed information is

incorporated in the system description/input files to the system analysis tool, identified in

Step #4 as the Aircraft Life Cycle Cost Analysis (ALCCA) [Galloway, Mavris, 1993].

Step #5 generates a joint probability distribution for each system, using Scheme #I.

10,000 sample points are generated for each aircraft by the system analysis tool through a

Monte-Carlo simulation. All four distributions are superposed in a two-dimensional joint

probability plot displayed for ROI and TOC in Figure 6.20 and for REV and TOC in

Figure 6.21. Since the Supersonic Transport has a significantly larger operating cost and

revenue than the other systems, and consequently skewing the scale, magnifications of

the joint probability plots for the B-747, B-777, and A340 are provided in Figures 6.22

and 6.23.

Figure 6.20: Joint Probability Plot forROI and TOC

Figure 6.21: Joint Probability Plot forREV and TOC

Area ofInterest

B-747

B-777

A340

Supersonic Transport Area ofInterest

B-747

B-777

A340

SupersonicTransport

(M$/yr)

(M$/

yr)

(M$/yr)


Figure 6.22: Magnified Joint ProbabilityPlot for ROI and TOC

Figure 6.23: Magnified Joint ProbabilityPlot for REV and TOC

The areas of interest indicated in the joint probability plots of Figures 6.20 to 6.23

are identified in Step #6, defined by maximum and minimum values for each criterion

that need to be satisfied. The return on investment should be as large as possible, i.e.

ROImax = ∞, but not smaller than 15% (ROImin). The revenue is also desired to be as large

as possible (REVmax = ∞), but should not be smaller than $100 million per year (REVmin).

The total operating cost, on the other hand, is supposed to be as small as possible, i.e.

TOCmin = $0, and should not be larger than $90 million per year (TOCmax). No

preferences are identified in Step #7 for the initial estimation of the Probability of

Success, which is finally calculated for each alternative in Step #8. The results for each

aircraft are summarized in Table 6.7, including the univariate probability of satisfying

just one criterion (disregarding the other criteria).

Area ofInterest

B-747

B-777

A340

(M$/

yr)

(M$/yr)

Area ofInterest

B-747

B-777

A340

(M$/yr)


Table 6.7: Summary of POS for each Alternative

Alternatives POS P(ROI > 15%) P(REV > M$100) P(TOC < M$90)

Boeing B-747 0.1379 0.7091 0.1391 0.9112

Boeing B-777 0.003 0.8363 0.003 1

Airbus A340 0 0.6960 0 1

SuperS. Transport 0 0.9097 1 0

Based on these results, the Boeing B-747 clearly yields the highest probability of

satisfying all criteria concurrently and must hence be considered the system of choice for

the airline. Had the decision been based on the return on investment or revenue alone,

the Supersonic Transport would be the best system. Had the decision been based on the

operating cost alone, the Airbus A340 would be the system of choice (see Figures 6.22

and 6.23 for discrimination between B-777 and A340). But since the decision is based on

all three criteria concurrently, the univariate probability values yield conflicting results:

choose the Supersonic Transport or the Airbus A340. Note that the Boeing B-747 does

not even appear as a viable alternative when the univariate probability values are used for

product selection, since there is always an alternative with a higher probabilities.

Furthermore, comparing this result to the result an Overall Evaluation Criterion

yields, the B-747 again does not appear to be the best alternative. As a matter of fact, by

using the common formulation with equal preference weights for the OEC,

)90$M(P)100$M(P%)15(POEC 31

31

31 <+>+>= TOCREVROI , (6.2)

the Supersonic Transport (OEC = 0.6366) turns out to be the best solution, followed by

the B-777 (OEC = 0.6131). The B-747 only comes in third with OEC = 0.5865, while


the A340 is with OEC = 0.5653 supposedly the worst solution for the airline.

Recognizing that the POS models the decision process of the airline more accurately

leads, however, to the conclusion that the OEC is not the best technique for a product

selection problem.

For resolving conflicts in product selection problems, like the one described

before, the Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS)

[Hwang, 1981] has been indicated to be a valuable decision making tool (see discussion

in Chapter II). In order to test the viability of TOPSIS in this product selection problem,

its result, using the univariate probability values from Table 6.7 is compared to the one

derived with the Joint Probabilistic Decision Making technique. First, TOPSIS requires a

normalization of the elements in the decision matrix:

=

019097.0

10696.0

1003.08363.0

9112.01391.07091.0

D and

=⇒=∑ =

09905.05737.0

5944.004389.0

5944.00030.05274.0

5416.01378.04472.0

4

1

2R

x

xr

j ji

jiji

. (6.3)

Next the Euclidean Distance to the ideal solution ri*, i.e. maximum value for each

criterion, and negative ideal solution ri-, i.e. minimum value for each criterion, is

calculated for each system. No preferences have been identified for this product selection

problem and all weights are equal to 1/3.


1981.033

3332.033

3295.033

2879.033

3

1

2*

*

3

1

2*340

*340

3

1

2*777

*777

3

1

2*747

*747

=

−=

=

−=

=

−=

=

−=

∑

∑

∑

∑

=

=

=

=

i

iSSTiSST

i

ii

i

ii

i

ii

rrS

rrS

rrS

rrS

3332.033

1981.033

2003.033

1863.033

3

1

2

3

1

2

340340

3

1

2

777777

3

1

2

747747

=

−=

=

−=

=

−=

=

−=

∑

∑

∑

∑

=

−

−

=

−

−

=

−

−

=

−

−

i

iSSTiSST

i

ii

i

ii

i

ii

rrS

rrS

rrS

rrS

(6.4)

Using the two Euclidean Distances, the Closeness Criterion is evaluated for each

system: ⇒−

=−

−

jj

jj SS

SC

*

C747 = 0.3929, C777 = 0.3781, C340 = 0.3729, CSST = 0.6271.

Based on TOPSIS, the Supersonic Transport is clearly the best system and should

be purchased by the airline. However, this solution is completely ignoring the fact that

the Supersonic Transport has no chance of ever meeting the operating cost requirement.

The JPDM technique had determined a POS of 0 for precisely this reason. Hence, it must

be concluded that TOPSIS is not a suitable technique for this product selection problem.

Its use and a decision based on it would result in high operating costs for the airline,

which its business structure may not be able to bare.

On the other hand, the airline may be attracted to the Supersonic Transport by its

high revenue and return on investment values and willing to make the higher investment

in operating cost. Then again, a smaller airline may not be able to afford such high

operating costs and will seek a system with lower cost and consequently lower revenues,

such as the Airbus A340. Without oversimplifying the problem, it can be assumed that

any airline would seek a system that yields a higher annual revenue than operating cost.


This behavior is signified by the new area of interest displayed in the joint probability

plot presented in Figure 6.24. The system that demonstrates the largest overlap of its

joint distribution with this area of interest yields the highest new Probability of Success

and should be purchased by the airline. This type of area of interest cannot be

accommodated by the current version of the Joint Probabilistic Decision Making

technique and is suggested as an area of future work at this point.

Figure 6.24: Joint Probability Plot for REV and TOC with New Area of Interest

While the Boeing B-747 turned out to be the best system for an airline which

requires the specified return on investment, revenue, and operating cost values, the actual

POS of 0.1124 is relatively small. The investigation into increasing this joint Probability

of Success is called a Requirement Trade-Off analysis and is demonstrated in the next

section.

A340

New AreaOf Interest

B-777B-747

(M$/

yr)

(M$/yr)


Requirements Trade-Off Analysis and Discussion

The main function of a requirement analysis is to produce insight into the decision

making problem by investigating the impact of requirements on the solution to the

decision making problem. As outlined in the flowchart in Figure 6.19, once the POS has

been determined for each system, requirement/criterion values may have to change, if the

POS values turned out to be too small.17 In order to increase the joint probability,

requirements have to be relaxed, i.e. made less stringent for the system to reach values of

interest. Relaxing the return on investment, however, has no effect on POS for any

system, as demonstrated in Table 6.8 with relaxed ROImin values of 10% and 5%. Only

the univariate probability of satisfying ROI increases, when compared to the values for

ROI > 15%. This can be attributed to the fact that the requirements for the revenue and

operating cost are far more stringent than the requirement for the return on investment.

Table 6.8: Comparison of POS for Different ROImin Values

BoeingB-747

BoeingB-777

AirbusA340

SupersonicTransport

POS(ROI > 15%, REV > M$100, TOC < M$90) 0.1124 0.003 0 0

POS(ROI > 10%, REV > M$100, TOC < M$90) 0.1124 0.003 0 0

POS(ROI > 5%, REV > M$100, TOC < M$90) 0.1124 0.003 0 0

P(ROI > 15%) 0.7091 0.8363 0.6960 0.9097

P(ROI > 10%) 0.8763 0.9323 0.8591 0.9693

P(ROI > 5%) 0.9604 0.9796 0.9461 0.9932

17 Note that technology infusion or any measure that changes the system is impossible in a productselection scenario, since the systems are fixed in their control variable values and noise variabledistributions.


Relaxing the revenue requirement has a much larger impact on the solution to the

product selection problem. Moreover, when relaxing the revenue requirement, the

operating cost requirement has to be tightened, since it is not practical for the airline to

consider systems that have a high chance of generating little revenue but high operating

costs. At the very least, the revenue should be equal or higher than the operating cost,

similar to the joint probability plot of Figure 6.24. Leaving ROImin at 15%, the graph in

Figure 6.25 examines how POS values change for the four different systems when

REVmin and TOCmax are changing concurrently.

0

0.1

0.2

0.3

0.4

0.5

0.6

50 100 150 200 250 300

REVmin (M$/yr) = TOCmax (M$/yr)

PO

S

SST

B-747

B-777

A340

Figure 6.25: Comparison of POS as a Function of Revenue and Cost Requirements

For airlines that can only afford a small average total operating cost, $68 million

per year or less, the Airbus A340 is clearly the best solution. For airlines with a slightly

larger pocket book, the Boeing B-777 becomes the best alternative. Note that the B-777

also has the highest absolute joint probability value of all four systems. If an even larger

aircraft is needed, i.e. Boeing B-747, the airline will have to sustain an even higher

operating cost, but is also guarantied a larger revenue to cover the cost. Finally, the


graph in Figure 6.25 denotes that the Supersonic Transport will only be a viable option

for very large airlines that can absorb operating costs of around $200 million per year and

aircraft.

For illustration purposes, Figures 6.26 through 6.29 display the joint probability

distributions for the four systems, in two dimensions (REV and TOC), at their respective

best requirement settings, identified in Figure 6.25. The Airbus A340 is the best solution

with a maximum Probability of Success of 0.4068 at REVmin = TOCmax = $66 million.

The Boeing B-777 is the system of choice with a maximum Probability of Success of

0.5713 at REVmin = TOCmax = $73 million, while the Boeing B-747 is the best system

with a maximum Probability of Success of 0.4363 at REVmin = TOCmax = $89 million.

Finally the Supersonic Transport is the best choice for the airline with a maximum

Probability of Success of 0.4492, if it can sustain operating costs of $202 million.

50 60 70 80 90 10040

50

60

70

80

90

100

110

120

130

TOC

RE

V

Figure 6.26: Joint Probability Plot forREV and TOC (A340 is best)

Figure 6.27: Joint Probability Plot forREV and TOC (B-777 is best)

Area ofInterest

B-747

B-777

A340

Area ofInterest

B-747

B-777

A340

(M$/

yr)

(M$/yr)

(M$/

yr)

(M$/yr)


Figure 6.28: Joint Probability Plot forREV and TOC (B-747 is best)

Figure 6.29: Joint Probability Plot forREV and TOC (SST is best)

Referring back to the flow chart in Figure 6.19, once the area of interest has been

fixed or if no change is desired in the first place, preferences can be changed among the

criteria. A change in preferences, i.e. a change in the preference weights of the criteria,

will change the POS value. However, the probability of achieving the criterion values

specified in Steps #6 and #10 of the flow chart stays the same. The weights merely alter

the target values according to Equations 5.31 and 5.32, which are subsequently used in

the calculation of POS (Equations 5.33 and 5.34). The probability of achieving the

specified criterion values is only equal to the POS value when all weights are equal, i.e.

no preferences are identified among the criteria. In order to understand the impact of

preferences on the solution to the product selection problem for the airline, POS values

for different preference weights are listed and compared in Table 6.9. All POS values are

based on the original criterion values of ROImax = ∞, ROImin = 15%, REVmax = ∞,

REVmin = $100 million, TOCmin = $0, and TOCmax = $90 million per year.

Area ofInterest

B-747

B-777

A340

Area ofInterest

B-747

B-777

A340

SupersonicTransport

(M$/

yr)

(M$/yr)

(M$/

yr)

(M$/yr)


Table 6.9: Comparison of POS Values for Different Criterion Preferences

Preference WeightswROI wREV wTOC

BoeingB-747

BoeingB-777

Airbus A340SupersonicTransport

1/3 1/3 1/3 0.1124 0.003 0 0

0.4 0.3 0.3 0.4563 0.0660 0.0006 0

0.5 0.25 0.25 0.2986 0.5056 0.1337 0

0.3 0.3 0.4 0.0186 0.0576 0.0006 0

0.25 0.25 0.5 0 0.0015 0.0123 0

A slight increase of the importance or preference of the return on investment does

not have any impact of the product selection outcome: the Boeing B-747 is still the best

system for the airline. However, increasing the preference significantly, e.g. ROI is twice

as important than the revenue and operating cost, changes the outcome and the Boeing

B-777 is the system of choice. Despite the fact that the B-777 is the best system in this

case, its probability of satisfying ROI > 15%, REV > $100 million, and TOC < $90

million is still only 0.003 = 0.3%. An increase in preference of the operating cost has an

even more drastic effect on the solution. A slight increase changes the best solution from

the B-747 to the B-777. If the preference of TOC is increased even further, e.g. twice as

important as the return on investment and revenue, the Airbus A340 becomes the best

solution for the airline. However, as Table 6.9 indicates, the A340 has no chance of

satisfying the specified criterion values. In conclusion, no matter what the preference

structure, the B-747 is the best choice for the airline in this example, given the

requirements of ROI > 15%, REV > $100 million, and TOC < $90 million, since all other

systems indicate no chance of satisfying these values concurrently.


Finally, in order to get a better feel for the sensitivity of POS with respect to

changes in the input distributions, a sensitivity analysis is conducted which determines

the impact of changes in noise variable means and standard deviations on POS. First,

new distributions are defined for the noise variables, which are summarized and listed

together with the old distributions in Table 6.10.

Table 6.10: Old and New Noise Variable Distributions

Noise Variable OldDistribution

Old Meanand Std. Dev.

NewDistribution

New Meanand Std. Dev.

Load FactorBeta

α = 3, β = 3µ = 0.75

σ = 0.037796Beta

α = 2, β = 4µ = 0.716667σ = 0.035635

Average YieldBeta

α = 5, β = 3µ = 0.1275

σ = 0.009682Beta

α = 3, β = 3µ = 0.12

σ = 0.011339

Fuel PriceBeta

α = 3, β = 5µ = 0.7125

σ = 0.048412Beta

α = 3, β = 3µ = 0.75

σ = 0.056695

Economic RangeBeta

α = 2, β = 4µ = 3666.67

σ = 356.3483Beta

α = 3, β = 3µ = 4000

σ = 377.9645

UtilizationBeta

α = 3, β = 3µ = 5000

σ = 188.9822Beta

α = 2, β = 4µ = 4833.3333σ = 178.1742

% Subsonic CruiseBeta

α = 3, β = 3µ = 0.25

σ = 0.094491Beta

α = 4, β = 2µ = 0.333333σ = 0.089087

InflationBeta

α = 3, β = 5µ = 0.04

σ = 0.01291Beta

α = 3, β = 3µ = 0.05

σ = 0.015119

Aircraft PriceBeta

α = 3, β = 3µ = 250

σ = 18.89822Beta

α = 3, β = 5µ = 237.5

σ = 16.13743

Passenger Weight Normalµ = 180σ = 10

Betaα = 4, β = 2

µ = 190σ = 10.69045

Baggage Weight Normalµ = 44σ = 5

Betaα = 2, β = 4

µ = 39σ = 5.345225

Using the ten-step process, outlined in Figure 6.19, new POS values are

determined for each of the four aircraft, by changing the distribution for one noise


variable at a time. The metrics by which the impact is assessed are based on a percentage

change of mean and standard deviation. The change in POS is not normalized, i.e. the

change is not a percentage but rather an absolute value, since a change in probability

seems more insightful than a percentage change with respect to some arbitrary value.

Both metrics are formulated as:

newoldnew

oldnew POSPOSPOS

µµµµ /)(% −−

=∂∂

(6.5)

newoldnew

oldnew POSPOSPOS

σσσσ /)(% −−

=∂∂

(6.6)

Using this formulation for the metrics, the sensitivities are obtained for each of

the four systems. For reasons discussed before, the POS of the A340 and the Supersonic

Transport is equal to zero and not affected by any changes in the noise variable

distributions. Consequently, Table 6.11 only lists the old and new POS values and

sensitivities for the B-747 and B-777. To further visualize the results from Table 6.11,

plots for the sensitivities with respect to the mean and standard deviation are presented in

Figures 6.30 and 6.31 respectively. Both the sensitivity plots and the results from

Table 6.11 clearly indicate that changes in the Load Factor, Average Yield, and

Utilization distributions have the largest impact on POS. This is true for both metrics,

Equations 6.5 and 6.6, as well as for both systems, B-747 and B-777. For example, a 1%

increase in mean for the Load Factor will increase the POS for the B-747 by 0.0176, a

12.8% increase with respect to the original POS value of 0.1379. Changes in POS are

smaller for the B-777, because its absolute POS values are small.


Table 6.11: Old and New POS Values and Sensitivities for B-747 and B-777

Change inDistribution

POS forB-747

∂POS∂%µ

∂POS∂%σ

POS forB-777.

∂POS∂%µ

∂POS∂%σ

Original 0.1379 N/A N/A 0.0030 N/A N/A

Load Factor 0.0597 0.0176 0.0137 0.0004 0.0006 0.0005

Average Yield 0.0654 0.0123 -0.0042 0.0011 0.0003 -0.0001

Fuel Price 0.1488 0.0021 0.0007 0.0025 -0.0001 -0.0000

Economic Range 0.1496 0.0013 0.0019 0.0037 0.0001 0.0001

Utilization 0.0793 0.0176 0.0102 0.0006 0.0007 0.0004

% Subsonic Cruise 0.1453 0.0002 -0.0013 0.0023 -0.0000 0.0000

Inflation 0.1436 0.0002 0.0003 0.0023 -0.0000 -0.0000

Aircraft Price 0.1458 -0.0016 -0.0005 0.0028 0.0000 0.0000

Passenger Weight 0.1448 0.0012 0.0010 0.0031 0.0000 0.0000

Baggage Weight 0.1385 -0.0001 0.0001 0.0019 0.0001 -0.0002

0.1496

0.0597

0.0037

0.0004

LF YIELD FUEL$ ECR U SUBS INFL PRICE PAXW BAGW

Figure 6.30: Sensitivity Plot for Changes in Mean for the B-747 and B-777

0.1496

0.0597

0.0037

0.0004

LF YIELD FUEL$ ECR U SUBS INFL PRICE PAXW BAGW

Figure 6.31: Sensitivity Plot for Changes in Standard Deviation for the B-747 and B-777

Particularly, the results for the metric based on changes in input means are not

very surprising and could have been obtained form a deterministic sensitivity analysis:

Load Factor, Average Yield, and Utilization are all variables that strongly affect the


revenue of the airline, which is the most stringent requirement for the B-747 and B-777

(see Table 6.7) that drives the value for POS most. Consequently, it is also not very

surprising that changes in POS due to changes in the input variables’ standard deviation

are mainly contributed to Load Factor, Average Yield, and Utilization. Slightly more

interesting is the direction of change. For example, POS decreases with increasing values

for the standard deviation of the average yield. This can be attributed to the fact that an

increase in Average Yield standard deviation allocates more revenue values outside the

area of interest. In contrast, the reverse is true for the Load Factor and Utilization,

demonstrated by the sensitivity plot in Figure 6.31.

Example Equation System with Ten Criteria

In order to demonstrate the capability of the Joint Probabilistic Decision Making

technique to handle more than two criteria, a system of ten equations is created for this

optimization example. The ten equations yield values for ten criteria that are to be

maximized. They depend on two design variables, x1, x2, and ten noise/uncertain

variables, Y1, Y2, Y3, Y4, Y5, Y6, Y7, Y8, Y9, Y10. All equations have a different optimum,

and therefore present a system of conflicting criteria. Specifically, the ten equations are:

212

2101101211 )()(200),,,( YxYxYYxxf −−−−= (6.7)

222

29192212 )()(200),,,( YxYxYYxxf −−−−= (6.8)

232

28183213 )()(200),,,( YxYxYYxxf −−−−= (6.9)

242

27174214 )()(200),,,( YxYxYYxxf −−−−= (6.10)


252

26165215 )()(200),,,( YxYxYYxxf −−−−= (6.11)

262

25165216 )()(200),,,( YxYxYYxxf −−−−= (6.12)

272

24174217 )()(200),,,( YxYxYYxxf −−−−= (6.13)

282

23183218 )()(200),,,( YxYxYYxxf −−−−= (6.14)

292

22192219 )()(200),,,( YxYxYYxxf −−−−= (6.15)

2102

2111012110 )()(200),,,( YxYxYYxxf −−−−= . (6.16)

The noise variables are normally distributed with a standard deviation of 1 but

different means, namely Y1 = N(1,1), Y2 = N(4,1), Y3 = N(5,1), Y4 = N(7,1), Y5 = N(8,1),

Y6 = N(10,1), Y7 = N(9,1), Y8 = N(6,1), Y9 = N(3,1), Y10 = N(2,1). A visual representation

of the ten equations, evaluated at the noise variable means, is presented in Figure 6.32,

highlighting the conflict in finding a single point that maximizes all functions. This fact

is supported also by the graph in Figure 6.33, displaying the ten different maxima of the

equations.

Figure 6.32: Surfaces of Ten Equations

0 2 4 6 8 100

2

4

6

8

10

x1

x2

Figure 6.33: Maxima of Ten Equations


Since the ten criteria depend on random variables, they themselves are random

variables as well. That means, the optimization problem of maximizing the ten equations

becomes a probabilistic optimization problem, maximizing the probability of all criteria,

Fi, being larger than a specified value. The value selected for this example is 170.

Hence, the optimization problem can be formulated as 21 ,

maxxx

P(F1 > 170, F2 > 170,

F3 > 170, F4 > 170, F5 > 170, F6 > 170, F7 > 170, F8 > 170, F9 > 170, F10 > 170).

However, this optimization problem can only be solved with the Joint Probability

Decision Making technique, since it requires a joint probability for ten criteria. In order

for the JPDM technique to be compared to other multi-criteria optimization methods, the

optimization problem has to be reformulated to:

)170)()(200(max)170),,,((max 212

2101

,101211

, 2121

>−−−−=> YxYxPYYxxFPxxxx

(6.17)

and )170)()(200(max)170),,,((max 222

291

,92212

, 2121


(6.18)

and )170)()(200(max)170),,,((max 232

281

,83213

, 2121


(6.19)

and )170)()(200(max)170),,,((max 242

271

,74214

, 2121


(6.20)

and )170)()(200(max)170),,,((max 252

261

,65215

, 2121


(6.21)

and )170)()(200(max)170),,,((max 262

251

,65216

, 2121


(6.22)

and )170)()(200(max)170),,,((max 272

241

,74217

, 2121


(6.23)

and )170)()(200(max)170),,,((max 282

231

,83218

, 2121


(6.24)

and )170)()(200(max)170),,,((max 292

221

,92219

, 2121


(6.25)

and )170)()(200(max)170),,,((max 2102

211

,1012110

, 2121


,(6.26)


which is not exactly the same problem, but the closest current multi-criteria optimization

methods can accommodate. The Utility Function and Goal Attainment methods are

selected for comparison with the JPDM technique.18 The Utility Function method,

similar to the JPDM technique, does not include an optimization algorithm itself. It is

simply comprised of an Overall Evaluation Criterion and is therefore used in conjunction

with a simplex search algorithm. In order to be able to compare the results, no

preferences are identified for the functions. Thus, the Utility Function and JPDM

technique are assigned equal weights, while the Goal Attainment method is assigned

weights of the same value as the goals, which are equal to the highest probability values

obtainable: One. The optimization problem can thus be formulated for the three

techniques as:

)170,170,170,170,170

,170,170,170,170,170(maxmax :JPDM

11176

54321,, 2121

>>>>>

>>>>>=

FFFFF

FFFFFPPOSxxxx

(6.27)

))170(1.0)170(1.0)170(1.0)170(1.0

)170(1.0)170(1.0)170(1.0)170(1.0

)170(1.0)170(1.0(maxmax :FunctionUtility

10987

6543

21,, 2121

>⋅+>⋅+>⋅+>⋅+

>⋅+>⋅+>⋅+>⋅+

>⋅+>⋅=

FPFPFPFP

FPFPFPFP

FPFPOECxxxx

(6.28)

Goal Attainment:21 ,

minxxλ (6.29)

subject to: 1)170( 1 ≥+> λFP and 1)170( 2 ≥+> λFP

and 1)170( 3 ≥+> λFP and 1)170( 4 ≥+> λFP and 1)170( 5 ≥+> λFP

and 1)170( 6 ≥+> λFP and 1)170( 7 ≥+> λFP and 1)170( 8 ≥+> λFP

and 1)170( 9 ≥+> λFP and 1)170( 10 ≥+> λFP .

18 The Minimax method was dropped, since it produced similar solutions to the Goal Attainment method inthe previous example in Chapter II.


To illustrate the use of Equations 6.27 and 6.28 as objective functions for

optimization and in order to get a better feel for the nature of the objective functions in

this example, a surface plot is displayed for both equations in Figures 6.34 and 6.35

respectively. Note that the Probability of Success surface has a much more pronounced

maximum than the Overall Evaluation Criterion surface. Even though both surfaces may

have similar optimal points, the OEC surface is much flatter, and hence slower for an

optimization algorithm to search (see previous optimization example).

Figure 6.34: POS Response Surface Figure 6.35: OEC Response Surface

The solutions to Equations 6.27, 6.28, and 6.29 are presented in Figure 6.36,

illustrating the differences in methods very well. The solutions to the ten probabilistic,

univariate optimization problems of maximizing P(F1 > 170), P(F2 > 170), P(F3 > 170),

P(F4 > 170), P(F5 > 170), P(F6 > 170), P(F7 > 170), P(F8 > 170), P(F9 > 170), or

P(F10 > 170) are individually added to the graph for reference purposes. The two

solutions generated by the Joint Probabilistic Decision Making technique correspond to

the two optimization techniques employed in conjunction with POS: a sequential

programming/line search and a simplex search technique.[Branch, Grace, 1996] All


solutions are found starting at (x1, x2) = (5, 5). However, some optimizations cannot find

a solution that improves the objective function value with respect to the starting point.

For these optimizations the starting point is the best solution.

1 2 3 4 5 6 7 8 9 101

2

3

4

5

6

7

8

9

10

x1

x2

Figure 6.36: Optimal Solutions

The true test of superiority for all methods is presented in Table 6.12. All

solutions identified in Figure 6.36 are listed with their corresponding joint probability of

achieving function values larger than 170, an OEC value based on Equation 6.28, as well

as their univariate probabilities. For ease of identification, the box with the objective

function value that was maximized in that optimization is highlighted. Clearly JPDM in

conjunction with the simplex search method yields the solution with the highest joint

JPDM/Simplex Search

(5.4502, 5.4330)

JPDM/SQP/Line Search

(5, 5)

Goal Attainment(5, 5)

OEC

(5.6422, 5.7019)

max P(F7 > 170)

(5.3281, 7.8594)

max P(F6 > 170)

(6.3516, 9.6172)

max P(F5 > 170)

(9.6172, 6.3516)

max P(F4 > 170)

(7.8594, 5.3281)

max P(F1 > 170)

(3.4844, 2.1875)

max P(F10 > 170)

(2.1875, 3.4844)

max P(F2 > 170)(5, 5)

max P(F3 > 170)(5, 5) max P(F8 > 170)

(5, 5)

max P(F9 > 170)(5, 5)


probability. The JPDM/line search solution has a lower joint probability, since the line

search, unlike the simplex search, does not find a point with improved POS with respect

to the starting point (5, 5). As before in previous examples, the OEC finds a solution that

falls just short of the joint probability value of the JPDM/simplex search solution. On the

other hand, it does find the solution with the highest OEC value, which supports the

previous argument that Equations 6.17 to 6.26 are simply not the right objective functions

for a joint probabilistic optimization problem. The Goal Attainment method cannot find

any solution, since it tries to achieve the same values for all univariate probabilities P(Fi

> 170). Consequently, it returns the starting point as its solution and it must be

concluded that this method is clearly not suitable for solving probabilistic multi-criteria

optimization problems. The remaining optimizations in Table 6.12 are merely

demonstrating that indeed maximizing just one objective P(Fi > 170) yields a small or no

probability of satisfying all criteria concurrently. As a matter of fact, the solutions to

max P(F2 > 170), max P(F3 > 170), max P(F8 > 170), and max P(F9 > 170) only yield a

significant amount of joint probability, because the starting point already produces the

maximum objective function value of 1. Therefore, no improvement is necessary, and

the starting point is determined the best solution, which happens to yield a joint

probability of 0.1790.

Table 6.12: Comparison of Joint and Univariate Probabilities for Different Objective Functions

JointProbability

OECValue

UnivariateP(F1≥170)










JPDM/Simplex Search 0.2170 0.7801 0.4040 0.9970 1.0000 0.9360 0.5640 0.5620 0.9350 1.0000 0.9970 0.4060

JPDM/SQP/Line Search 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420

OEC 0.1950 0.7898 0.3010 0.9950 1.0000 0.9700 0.6780 0.6840 0.9700 1.0000 0.9940 0.3060

Goal Attainment 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420

max P(F1 ≥ 170) 0 0.5885 0.9990 1.0000 0.9480 0.0260 0.0010 0 0.0070 0.9060 1.0000 0.9980

max P(F2 ≥ 170) 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420

max P(F3 ≥ 170) 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420

max P(F4 ≥ 170) 0.0060 0.7208 0.0360 0.6630 1.0000 1.0000 0.9720 0.7600 0.9420 0.9960 0.8250 0.0140

max P(F5 ≥ 170) 0 0.5747 0 0.0470 0.9320 1.0000 1.0000 0.9090 0.9460 0.7850 0.1280 0

max P(F6 ≥ 170) 0 0.5747 0 0.1280 0.7850 0.9460 0.9090 1.0000 1.0000 0.9320 0.0470 0

max P(F7 ≥ 170) 0.0060 0.7208 0.0140 0.8250 0.9960 0.9420 0.7600 0.9720 1.0000 1.0000 0.6630 0.0360

max P(F8 ≥ 170) 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420

max P(F9 ≥ 170) 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420

max P(F10 ≥ 170) 0 0.5885 0.9980 1.0000 0.9060 0.0070 0 0.0010 0.0260 0.9480 1.0000 0.9990

Oliver B

andte C

hapter VI

165

Oliver Bandte Chapter VII 166

CHAPTER VII

CONCLUSIONS

It has always been the intention of systems engineering to invent or produce the

best product possible. Many design techniques have been introduced over the course of

decades that try to fulfill this intention. Some addressed the problem of capturing the

customer’s desires and needs. Others addressed the inability of accurate deterministic

performance and cost prediction caused by the inherent uncertainty in assumed values for

the parameters the prediction is based on. Unfortunately, no technique succeeded in

addressing both issues concurrently. The work presented in this thesis successfully

overcomes this deficiency by developing a multi-criteria decision making with

probabilistic design. Its core development is a multivariate probability distribution that

serves in conjunction with a criterion value range of interest as a universally applicable

objective function for multi-criteria optimization and product selection. This objective

function produces a meaningful metric, Probability of Success, that allows the customer

or designer to make a decision based on the chance of satisfying the customer’s goals.

Joint Probabilistic Decision Making Technique in Design

The design technique developed in this thesis is called the Joint Probabilistic

Decision Making (JPDM) technique. It was built cognizant of the fact that systems

design, at its core, is a decision making process in which the individual decisions concern


the allocation of resources and finding the right values for the design parameters. It is

this process that JPDM facilitates when decisions are based on multiple criteria while the

values for particular design or environmental parameters are uncertain.

Most decision processes can be grouped into two types of problems: optimization

and product selection. Both problems can be solved by comparing Probability of Success

(POS) values for the different decision alternatives, i.e. POS becomes the objective

function for multi-criteria optimization or the product selection metric that allows for an

equal basis comparison of the alternatives. POS itself describes the likelihood that the

system will achieve criterion values of interest to the customer. It is calculated by

integrating the joint probability distribution for the criteria the decision is based on over

the area of criterion values that satisfy the customer’s requirements and desirements.

While the area of interest can be determined from the needs analysis in the early phases

of design, the determination of the joint probability distribution is less obvious and was

the subject of this research.

Algorithms for Determining the Joint Probability Distribution

To incorporate a joint probabilistic formulation into the systems design process,

algorithms need to be created that allow for an easy implementation into a numerical

framework. Rules were established as a guide to which algorithm is best suited for

implementation into systems design:

• Algorithms may not be limited in number of random variables, i.e. criteria.

• Algorithms need to be flexible with respect to the criterion distributions.


• Algorithms have to satisfy the conditions of a joint probability distribution.

• Algorithms cannot require numerical integration.

Unfortunately, most algorithms producing a joint probability distribution that can

be found in the literature either attempt to approximate data from a specific process or

accommodate just one type of distribution. Only a joint-normal formulation would be

compatible with the flexibility requirement, since transformations of actual univariate

criterion distributions could yield normal distributions, to be used in a joint-normal

algorithm. Unfortunately, the joint-normal algorithm is not defined for all combinations

of (negative) correlation coefficients and criterion variances. Even worse, since the

integral of the characteristic function for the (joint) Normal-Distribution does not exist

and numerical integration of the algorithm for a large number of criteria is prohibitively

expensive, the joint-normal formulation can only be used for a small number of criteria.

To overcome this lack of adequate algorithms for generating joint probability

distributions, two new algorithms were created for this research: the (multivariate)

Empirical Distribution Function and the Joint Probability Model. The non-parametric

Empirical Distribution Function (EDF) simply estimates the probability of occurrence of

an event by counting how many times that event occurred in a given sample. In

probabilistic systems design, the EDF has commonly been used in estimating the

univariate distributions of system characteristics from samples generated by a sampling

technique such as Monte-Carlo simulation. This thesis outlined an extension to a joint

(multivariate) formulation. The Joint Probability Model (JPM) on the other hand is an

parametric model to estimate the multivariate joint probability. It is comprised of the


product of the univariate criterion distributions, generated by the traditional probabilistic

design process, multiplied with a correlation function that is based on available

correlation information between pairs of random variables. The selection process for the

correlation function has also been outlined in this thesis. Finally, Table 7.1 summarizes

the advantages and disadvantages of the EDF and JPM.

Table 7.1: Advantages and Disadvantages of EDF and JPM

EDF JPM

Advantages

• No approximation withstandard distribution needed.

• Estimates probability fromdata directly.

• Exact in the limit.• Fast estimation of joint

probability.

• Only limited informationneeded.

• Can employ expert guesses.• Easy use in conceptual design.

Disadvantages

• Large amount of data neededin order to be accurate.

• Requires modeling andsimulation.

• Requires approximation withparametric distribution.

• Requires correlation function.• Estimation of joint probability

is time consuming.

JPDM for Optimization

The Joint Probabilistic Decision Making technique is a valuable tool for multi-

objective optimization, due to its ability to transform disparate objectives into a single

figure of merit, the likelihood of successfully meeting all goals, which is a single

optimizable function. This new objective function, called Probability of Success, allows

for the use of any standard single-objective optimization technique available. By

distinguishing between controllable and uncontrollable variables in the design process,


JPDM can account for the uncertain values of the uncontrollable variables that are

inherent to the design problem, while facilitating an easy adjustment of the controllable

ones to achieve the highest Probability of Success possible.

The application examples in this thesis demonstrated the technique’s ability to

produce a better solution with a higher Probability of Success than an Overall Evaluation

Criterion (OEC) or Goal Programming approach. The first example was an optimization

problem to determine a supersonic transport aircraft configuration that would maximize

the returns on investment for the manufacturer and the airline concurrently, subject to

landing and take-off field length and approach speed constraints. JPDM yielded a

solution with a higher Probability of Success (0.782) than the OEC (0.773). In addition,

the solution was found in 71 rather than the 109 iterations needed by the OEC.

The other application example employed a set of ten equations, function of two

design/controllable and ten noise/uncontrollable variables. All ten functions were

contrived such that the location of their maximum within the design space was dependent

on the setting of the noise variables. Consequently, the optimization problem of

concurrently maximizing all ten functions presented a conflict that allowed no

deterministic solution. However, it was shown that, upon assignment of probability

distributions to the noise variables, only the JPDM technique is able to solve this

problem, since it is the only technique capable of forming and treating a joint probability

distribution of the responses. A comparison of the JPDM solution to OEC and Goal

Programming on a makeshift problem, that intended to maximize the individual

probabilities concurrently rather than one joint distribution, demonstrated JPDM’s ability


to produce the best solution. The Goal Programming technique did not find any better

solution than the starting point while the OEC found a solution better than the starting

point, but with less POS than the JPDM solution.

JPDM for Product Selection

The Joint Probabilistic Decision Making technique is also an outstanding tool for

multi-attribute product selection, due to its ability to transform the likelihood of

achieving certain criterion values into a single metric that allows comparison of all

alternative solutions on an equal basis. This new metric, called Probability of Success

(POS), simplifies a complex multi-criteria selection problem into a simple ordering

problem, where the solution with the highest POS is the best solution. Also, by

distinguishing between controllable and uncontrollable variables in the design process,

JPDM can account for the uncertain values of the uncontrollable variables that are

inherent to the problem while distinguishing between alternatives through different

settings of the controllable ones. After deriving a joint criterion probability distribution

for each alternative, the Probability of Success can be obtained for each alternative by

integrating each distribution over the customer specified area of criterion values.

Subsequently, trade-off studies can be conducted at a very low level of effort that keep

the alternative distributions constant but change the criterion values of interest and

thereby change the Probability of Success for each alternative.

The application example in this thesis demonstrated the technique’s ability to

produce a better solution with a higher Probability of Success and different ranking than


the Overall Evaluation Criterion or Technique for Order Preferences by Similarity to the

Ideal Solution (TOPSIS) approach. This product selection problem determined an

airline’s best choice among the Boeing B-747, Boeing B-777, Airbus A340, and a

notional Supersonic Transport. The selection was based on values for return on

investment, average annual revenue, and total average annual operating cost. JPDM

determined the B-747 to be the best solution with the highest Probability of Success for

the scenario considered herein, while the OEC and TOPSIS both predicted the

Supersonic Transport to be the best selection for the airline. Considering that the

Supersonic Transport has a zero probability of satisfying all three criteria and their

respective values concurrently, this result has to be deemed erroneous and the JPDM

seems to be the only technique producing a sensible solution.

Upon further deliberation, the return of investment requirement was determined to

have very little influence on the best solution, since varying return of investment

requirement values produced the same result. However, changes in the requirements for

the revenue and total operating cost had a significant influence on the solution to the

selection problem, each aircraft being the best solution at some point, dependent on the

requirement value for revenue and operating cost. One conclusion drawn from these

results was that the current shape of the area of interest with independent requirement

values may not be adequate for all decision problems, and it was suggested to expand the

technique to include areas of interest with dependent criterion requirement values.


Research Questions and Answers

Finally, it is useful to consider again the research questions posed in the

introduction to this thesis and what this thesis has demonstrated in response to them.

• Does uncertainty in the systems engineering design obstruct the decision process?

− No. When treated properly, through probabilistic assumptions for example,

uncertainty about numerical values of design or operational parameters, model

fidelity, or technology availability does not impede the decision making process.

• Can the use of probabilistic design be beneficial in the decision making process?

+ Yes. As demonstrated in the thesis, multi-criteria optimization or product selection

problems that don’t have a deterministic solution can be solved through intelligent

use of a joint probability distribution.

• Is there a numerical value representing customer satisfaction?

+ Yes. The numerical value established in this thesis is called Probability of Success,

and is obtained by integrating the joint criterion distribution over the area of criterion

values that are of interest to the customer.

• Does a technique already exist that can help the decision-maker find a best solution

based on multiple criteria arising from a probabilistic design technique?

− No. An extensive search of the current literature brought about no reference that

indicated a particular technique that combines multi-criteria decision making with

uncertain criterion values.

• Is it possible to create such a technique and what should it look like?

+ Yes. As the thesis demonstrates, a joint probability distribution can be created for all

criteria of the multi-criteria decision problem, while accounting for the inherently

uncertain values of uncontrollable parameters in the design process.


• Can this technique be used for optimization?

+ Yes. As demonstrated in the thesis, the Probability of Success can serve as an

objective function that can be used by any optimizer to find the best solution,

maximizing customer satisfaction.

• Can this technique be used for product selection?

+ Yes. As demonstrated in the thesis, the Probability of Success can be used as a metric

that allows for a comparison of alternatives on an equal basis, the alternatives simply

need to be ranked based on their Probability of Success value.

Recommendations

In closing, an outlook for future research is provided at this point. It was

mentioned before and in Chapter VI, that in certain decision making situations

independent criterion value requirements for the area of interest may not be appropriate.

One modeling option to remedy the situation is to create linear equations of the criterion

requirement values zmin and zmax of the form:

01

maxmin/0 =+ ∑=

N

iii zaa , 0

1maxmin/0 =+ ∑

=

N

iii zbb , and so on. (7.1)

More complex shapes of the area of interest can be proposed as well, but its applicability

is questionable and can only be justified if detailed knowledge about the shape of the area

of interest is available. Some equations could be of the form:

01

2maxmin/0 =+ ∑

=

N

iii zaa , 0

1

2maxmin/0 =+ ∑

=

N

iii zbb , and so on, (7.2)

or 01

0maxmin/ =+ ∑

=

N

i

zi

ieaa , 01

0maxmin/ =+ ∑

=

N

i

zi

iebb , and so on. (7.3)


Since this thesis borders closely on the area of utility theory in decision making, a

possible and natural extension of this research is to include a criterion value utility in a

new formulation of the Probability of Success. This new formulation may favor design

solutions that produce criterion values far within the area of interest. But it is not clear at

this point whether this may also be achieved by simply adjusting the area of interest such

that solutions with a previously low Probability of Success drop out. Adjustment and use

of the formulation for preference weights as proposed in this thesis may achieve this goal

as well. The benefit of a utility function for criterion values would only be noticeable

when this function has a non-monotonic behavior that favors criterion values in a

particular region. And even in this case, an intelligent way of modeling the area of

interest may yield the same effect.

Finally, a comment on conditional probabilities is in order. The utilization of a

joint probability distribution as the central element in this new decision making technique

suggests the investigation into use of conditional probabilities as well. While this

investigation was not further documented in this thesis, it was found that the use of

conditional probability does not add to the decision making process and is confusing at

best. When using conditional probability to indicate the joint or univariate distribution

for one or more criteria with fixed values for the other criteria, the Empirical Distribution

Function collapses due to the sharply reduced sample size of the conditional probability.

Even opening up the range of the condition, i.e. P(x, y| z1 < z < z2) rather than

P(x, y| z = z0), decreases the sample size for the EDF to a point where it may no longer be

accurate. It can also be confusing to use a conditional probability for the plotting of the


(conditional) joint distribution for two criteria. While the Probability of Success for a

particular alternative may be small, conditioning the probability may display the

(conditional) joint distribution well within the area of interest, suggesting a high POS.

This confusing discrepancy is due to the fact that the criteria may be (strongly) correlated

and the process of eliminating the sample points with unfavorable values for the criterion

the distribution is conditioned on may actually eliminate the sample points with

unfavorable values for the criteria displayed. This would yield a probability distribution

that does not show any probability mass or density for unfavorable values (i.e. outside of

the displayed area of interest), yielding a (conditional) distribution well within the area of

interest. For these reasons the concept of conditional probability was not included in this

thesis and is probably not worth further investigation.

Oliver Bandte Appendix A 177

APPENDIX A

DISTRIBUTIONS

This appendix lists the parametric formulation of all univariate distributions

mentioned in this thesis. Formulations for these distributions can be found in any

probability theory reference, but have been taken for this appendix from [Hogg, Tanis,

1993] and [Jones, 1997] specifically. Many distributions are only defined for positive

values of the independent variable x. In order to be able to use the listed distributions in

systems design for criteria with non-positive values, modifications have been made to the

formulations that extend the range of valid x-values. Generally, a Min-Value and a Max-

Value limits this range. The range of validity for x is indicated for each distribution. α,

and β are distribution specific parameters that did not need modification from the

standard formulation.

Beta-Distribution, B(α, β, Min, Max):

11

)()(

),()(

−−

−−

−−

ΓΓΓ=

βα

βαβα

MinMax

xMax

MinMax

Minxxf , Min ≤ x ≤ Max (A.1)

Gamma-Distribution, G(α, β, Min):

βααβα

Minx

eMinxxf−−

−−Γ

= 1)()(

1)( , Min ≤ x ≤ ∞ (A.2)

Oliver Bandte Appendix A 178

Lognormal-Distribution, LN(µ, σ):

2

2

2

))(ln(

2

1)( σ

µ

πσ

−−=

x

ex

xf , -∞ ≤ x ≤ ∞ (A.3)

Normal-Distribution, N(µ, σ):

2

2

2

)(

2

1)( σ

µ

πσ

−−=

x

exf , -∞ ≤ x ≤ ∞ (A.4)

Uniform-Distribution, U(Min, Max):

MinMaxxf

−= 1

)( , Min ≤ x ≤ Max (A.5)

Weibull-Distribution, W(α, β, Min):

βαβαβ )(1)()( MinxeMinxxf −−−−= , Min ≤ x ≤ ∞ (A.6)

Oliver Bandte Appendix B 179

APPENDIX B

CORRELATION FUNCTIONS

The correlation function introduced in Chapter V, Equation 5.18, is particularly

simple and easy to apply. However, due to the nature of the function, it does not increase

any joint density values around the midpoint of the range it is defined over. It only

significantly increases density values over points located close to the two opposite

corners of that range (see graph in Figure B.1). This may not be much of a disadvantage

for such joint distributions as the one displayed in Figures 5.5 and 5.6. However,

functions with a mode close to the center point of the range are much more effected by

this skewed distribution of density. As a matter of fact, very high values for ρ can

produce a bimodal joint distribution and negative density values, invalidating the function

as a joint probability density function.

Figure B.1: Surface Plot of Equation 5.18 for Two Variables


A more suitable function is identified in Equation B.1 and B.2 for positive and

negative ρ. Both equations are displayed in the graphs of Figures B.2 and B.3.

2*2

*1

21 )(1

1),(

xxxxg

−+=

ρ, ρ > 0 (B.1)

2*2

*1

21 )(1

1),(

xxxxg

+−=

ρ, ρ < 0 (B.2)

with min1max1

mid11*1 2

xx

xxx

−−

= , min2max2

mid22*2 2

xx

xxx

−−

= , 2

min1max1mid1

xxx

+= , 2

min2max2mid2

xxx

+= ,

max11min1 xxx << , and max22min2 xxx << .

Figure B.2: Surface Plot of Equation B.1 Figure B.3: Surface Plot of Equation B.2

To visualize the impact of Equations B.1 and B.2, two univariate beta

distributions are employed: X1 ~ B(6,6,0,1) and X2 ~ B(6,6,0,1). Their joint distribution,

without any correlation, is displayed in Figure B.4. When employing Equation B.1 as a

correlation function with ρ = 0.09, points close to the x1 = x2 line see an increase in

likelihood of occurrence, as indicated in the joint probability plot of Figure B.5. This

trend is increased dramatically, when increasing ρ to 0.9, as demonstrated in the joint

probability plot of Figure B.6.


0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x1

x2

Figure B.4: Joint Probability Plot for Two Beta-Distributions (ρ = 0)

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x1

x2

Figure B.5: Joint Probability Plot forTwo Beta-Distributions with Correlation

Function B.1 (ρ = 0.09)

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x1

x2


Function B.1 (ρ = 0.9)

If other shapes are desired for the joint probability distribution, the correlation

function can be constructed from more basic functions. Equations B.3 through B.6, for

example, skew the distribution towards a particular corner. Their effect is demonstrated

with the previously mentioned univariate beta-distributions in the joint probability


contour plots of Figures B.7 through B.10. γ is a positive scalable parameter that

determines the skewness of the distribution.

g(x1, x2) = 1 + γ x1*x2

*,min1max1

min11*1 xx

xxx

−−

= ,min2max2

min22*2 xx

xxx

−−

= (B.3)

g(x1, x2) = 1 + γ x1*x2

*,min1max1

min11*1 xx

xxx

−−

= ,min2max2

2max2*2 xx

xxx

−−

= (B.4)

g(x1, x2) = 1 + γ x1*x2

*,min1max1

1max1*1 xx

xxx

−−

= ,min2max2

min22*2 xx

xxx

−−

= (B.5)

g(x1, x2) = 1 + γ x1*x2

*,min1max1

1max1*1 xx

xxx

−−

= ,min2max2

2max2*2 xx

xxx

−−

= (B.6)

with max11min1 xxx << , max22min2 xxx << , and γ > 0.

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x1

x2


Function B.3 (γ = 4.9)

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x1

x2




0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x1

x2



0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x1

x2



Oliver Bandte Appendix C 184

APPENDIX C

NUMBER OF SAMPLES FOR THE MONTE-CARLO

SIMULATION

The Monte-Carlo simulation is a sampling technique that determines which input

variable values to chose for a given simulation run. These input values are used in an

equation or analysis code, yielding the outputs of interest. Since the outputs depend on

random variables, they are random variables themselves. However, the Monte-Carlo

simulation does not determine the output distribution, but rather creates a random sample

X1, X2, X3, …, Xn from a Bernoulli distribution. In other words, each simulation run, or

sample i, determines, whether the analysis outcome, R, is less, more, or equal to a

specified value a; if so, Xi = 1, if not, Xi = 0.[Shooman, 1968]

Summing all samples yields a new random variable, Y = ΣXi, which is binomially

distributed with a mean µ = np and variance σ 2 = np(1-p). n is the number of samples

run in the Monte-Carlo simulation, and p is the true probability of the specified event, as

in p = P(R < a). The question then becomes how many sample cases are needed to bring

the estimator of p, Y/n, reasonably close to p.

The Central Limit Theorem states that the distribution of

σµ−= Y

W(C-1)


is N(0,1), i.e. normal with µ = 0 and σ2 = 1, in the limit where n Å ∞. Thus, if n is

sufficiently large, the binomial distribution of Y can be approximated with

N(np, np(1-p)). Sufficiently large is determined by experience more than anything else.

Typically, the condition np > 5 and n(1-p) > 5 is a good guideline.[Shooman, 1968] The

95% confidence interval for Y/n being close to p , i.e. W = 0, is

%95)202( =<−<− σσ WP . (C-2)

Since W is N(0, 1), the interval becomes P(-2 < W < 2) = 95%, or

%95))1(2)1(2( =−<−<−− pnpnpYpnpP . (C-3)

Dividing by n we have

%95))1(

2)1(

2( =−<−<−−n

ppp

n

Y

n

ppP , (C-4)

which is a 95% confidence interval for the estimator of p, Y/n. If the percent error with

respect to p is defined by

p

pnY −

=ε , (C-5)

combining it with Equation C-4 yields

np

p−= 12ε . (C-6)

Solving for n yields the equation that determines how many Monte-Carlo

simulation cases need to be run for the error ε to be within a 95% confidence interval at

the probability level p:


p

pn

−= 142ε

. (C-7)

From Equation C-7 and its depiction in Figure C-1 it follows that the number of

samples depends highly on the probability level of interest and ε. While the logarithmic

scale is deceiving, the parallel lines for different ε indicate that the number of samples, n,

increases for smaller values of ε much more rapidly with decreasing probability levels p.

For probability levels larger than 0.9 the number of samples becomes so small that the

sampling cannot be considered statistically significant anymore.

1

10

100

1000

10000

100000

1000000

10000000

0 0.2 0.4 0.6 0.8 1

Probability Level p

Nu

mb

er o

f S

amp

les

n

1%2%5%10%15%

Figure C.1: Number of Samples as a Function of Probability Level and Percent Error

% Error

Oliver Bandte Appendix D 187

APPENDIX D

JOINT PROBABILISTIC DECISION MAKING TECHNIQUE

- THE COMPUTER PROGRAM -

This appendix provides a list of the files created for this thesis work. Taken

together they comprise the computer program that calculates the Probability of Success

and creates the graphs displayed in Chapters V and VI. The flow of information within

the program is outlined in the flow chart of Figure D.1.

POS

JPDM.m inputs.m

EDF or JPM

contourplot.m

surfplot.m

cumprobplot.m

JPM.m

JPMprob.m

2 0 1 4 0 1 6 0 1 8 0 2 0 0 2 2 0 2 4 0 2 6 0 2 8 0 3 0

EDF.mEDF#.exe

…

passback

POS

Data

passEDF

Data1 Data2 DataM

- 5 0 5 1 05

0

5

0

5

callsreads increates

Figure D.1: Flow Chart of the Joint Probabilistic Decision Making Technique


The Joint Probabilistic Decision Making technique is currently coded in Matlab®.

The file that starts the process is called JPDM.m and is invoked from within Matlab®. Its

principal function is to read the input information from inputs.m into the memory and

make the decision based on the inputs whether to execute the Joint Probability Model

(JPM.m) or the Empirical Distribution Function (EDF.m). The Joint Probability Model

calculates the POS based on the distribution parameters and correlation function specified

in inputs.m. For visual checks of the inputs, JPM.m provides the ability to plot the

specified univariate distributions. If inputs.m indicates a request for plots,

JPMprob.m is called from JPM.m to calculate the probability (density) values required

for a two-dimensional contour plot (contourplot.m), three-dimensional surface plot

(surfplot.m), or two-dimensional cumulative probability contour plot

(cumprobplot.m).

In order to use the Empirical Distribution Function, data needs to be provided in

form of columns, one for each criterion, within files named Data1, Data2, …, DataM,

with M being the number of alternatives evaluated concurrently. For optimization, one

file, Data1, which changes with each iteration is sufficient. EDF.m will copy the data

for each alternative into a new file called Data, and supply information about the area of

interest, the size of Data, and step size for plotting in a file called passEDF. Next, it

calls EDF#.exe, “#” is the number of criteria, which in turn reads the information from

passEDF and Data. EDF#.exe is a short FORTRAN program that contains the actual

counting procedure for the Empirical Distribution Function. The decision to code this

procedure in FORTRAN was based on the general availability of FORTRAN and the fact


that the counting takes orders of magnitude longer in MATLAB®. EDF#.exe, then,

supplies joint and univariate probability values for the criteria as well as probability

values for plotting in a file called passback back to EDF.m. This loop is repeated for

all alternatives until EDF.m finally prints out the joint POS and univariate probability

values for all alternatives. If plotting is requested in inputs.m, contourplot.m,

surfplot.m, and/or cumprobplot.m are called to produce a two-dimensional

contour plot, three-dimensional surface plot, and/or two-dimensional cumulative

probability contour plot.

List of Files

JPDM.m

% JPDM.m %clear allff = clock;% Default Valuescrplots = [0 0];t = 1;sp = 0;aalow = [0 0 0 0];aahigh = [0 0 0 0];

inputsnfom = length(maxvalue);if method == 1 points = length(distribution(:,1)); JPMelseif method == 2 points = length(l); EDFend


inputs.m (Joint Probability Model)

% inputsmethod = 1;syms x1 x2 ;% Choose Distribution from:% 1 = Beta(a,b,U,L)% 2 = Exponential(mu)% 3 = negExponential(mu)% 4 = Gamma(a,b,L)% 5 = negGamma(a,b,U)% 6 = Lognormal(L,mu,sig)% 7 = negLognormal(U,mu,sig)% 8 = Normal(mu,sig)% 9 = Rayleigh(b,L)% 10 = negRayleigh(b,U)% 11 = Uniform(U,L)% 12 = Weibull(a,b,L)% 13 = negWeibull(a,b,U)% 14 = Student’s t(mu,a)distribution(1,1) = 1;fom(1) = x1;a(1,1) = 5;b(1,1) = 10;L(1,1) = 135;U(1,1) = 147 + L(1,1);lower(1,1) = 135;upper(1,1) = U(1,1);plotrange(1,:,1) = [L(1,1) U(1,1)];ipl(1,1) = ’y’;distribution(1,2) = 1;fom(2) = x2;L(1,2) = -5;U(1,2) = 111.5;a(1,2) = 8;b(1,2) = 21;lower(1,2) = -5;upper(1,2) = 111.5;plotrange(2,:,1) = [-10 80];ipl(1,2) = ’y’;distribution(2,1) = 1;a(2,1) = 6;b(2,1) = 7;L(2,1) = 69;U(2,1) = 91;lower(2,1) = 69;upper(2,1) = 91;plotrange(1,:,2) = [69 91];ipl(2,1) = ’y’;distribution(2,2) = 1;L(2,2) = -5;a(2,2) = 4;b(2,2) = 3;U(2,2) = 36;lower(2,2) = -5;


upper(2,2) = 36;plotrange(2,:,2) = [-10 80];ipl(2,2) = ’y’;distribution(3,1) = 1;a(3,1) = 6;b(3,1) = 8;L(3,1) = 55;U(3,1) = 81;lower(3,1) = 55;upper(3,1) = 81;plotrange(1,:,3) = [55 81];ipl(3,1) = ’y’;distribution(3,2) = 1;L(3,2) = -5;a(3,2) = 6;b(3,2) = 5;U(3,2) = 46.5;lower(3,2) = -5;upper(3,2) = 46.5;plotrange(2,:,3) = [-10 80];ipl(3,2) = ’y’;fcorr(1) = 1+0.41*(2*fom(1)-(upper(1,1)+lower(1,1)))/(upper(1,1)-

lower(1,1))*(2*fom(2)-(upper(1,2)+lower(1,2)))/(upper(1,2)-lower(1,2));

fcorr(2) = 1+0.0*(2*fom(1)-(upper(2,1)+lower(2,1)))/(upper(2,1)-lower(2,1))*(2*fom(2)-(upper(2,2)+lower(2,2)))/(upper(2,2)-lower(2,2));

fcorr(3) = 1+0.0*(2*fom(1)-(upper(3,1)+lower(3,1)))/(upper(3,1)-lower(3,1))*(2*fom(2)-(upper(3,2)+lower(3,2)))/(upper(3,2)-lower(3,2));

% Requirementsmaxvalue = [90 150];minvalue = [0 15];% Plot informationnstep = [50 50 50 50 50 50 50 50 ];crplots = [1 2 ];condrange = [-10 -10 500 100 ];probplot2D = ’y’;probplot3D = ’y’;cumplot = ’y’;cp = (0.00001:0.0002:0.5);cc = (0:0.1:.99);aalow = [50 -10 ];aahigh = [250 80 ];name(1) = ’TOC(M$/yr)’;name(2) = ’ROI’;contourcolors = [’k’ ’b’ ’r’ ’c’];


JPM.m

% JPM.mfor n = 1:points; for i = 1:size(distribution,2) if distribution(n,i) == 1 func(i) = gamma(a(n,i)+b(n,i))*((fom(i)-L(n,i))/(U(n,i)-

L(n,i)))^(a(n,i)-1)*((U(n,i)-fom(i))/(U(n,i)-L(n,i)))^(b(n,i)-1)/(gamma(a(n,i))*gamma(b(n,i)));

if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 2 func(i) = exp(-fom(i)/mu(n,i))/mu(n,i); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 3 func(i) = exp(fom(i)/mu(n,i))/mu(n,i); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 4 func(i) = (fom(i)-L(n,i))^(a(n,i)-1)*exp(-(fom(i)-L(n,i))/

b(n,i))/(b(n,i)^a(n,i)*gamma(a(n,i))); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:));


figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 5 func(i) = (U(n,i)-fom(i))^(a(n,i)-1)*exp(-(U(n,i)-fom(i))/

b(n,i))/(b(n,i)^a(n,i)*gamma(a(n,i))); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 6 func(i)=exp((log(fom(i)-L(n,i))-mu(n,i))^2/(2*sig(n,i)^2))/

((fom(i)-L(n,i))*sig(n,i)*sqrt(2*pi)); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(i) == 7 func(i)=exp((log(U(n,i)-fom(i))-mu(n,i))^2/(2*sig(n,i)^2))/

((U(n,i)-fom(i))*sig(n,i)*sqrt(2*pi)); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 8 func(i) = exp(-(fom(i)-mu(n,i))^2/

(2*sig(n,i)^2))/(sig(n,i)*sqrt(2*pi)); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n);


ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 9 func(i) = (fom(i)-L(n,i))*exp(-(fom(i)-L(n,i))^2/

(2*b(n,i)^2))/b(n,i)^2; if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 10 func(i) = (U(n,i)-fom(i))*exp(-(U(n,i)-fom(i))^2/

(2*b(n,i)^2))/b(n,i)^2; if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 11 func(i) = 1/(U(n,i)-L(n,i)); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 12 func(i) = b(n,i)*((fom(i)-L(n,i))/a(n,i))^(b(n,i)-1)*

exp(-((fom(i)-L(n,i))/a(n,i))^b(n,i))/a(n,i); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n);


ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 13 func(i) = a(n,i)*b(n,i)*(U(n,i)-fom(i))^(b(n,i)-1)*

exp(-a(n,i)*(U(n,i)-fom(i))^b(n,i)); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end elseif distribution(n,i) == 14 func(i)=gamma((a(n,i)+1)/2)/(sqrt(pi*a(n,i))*gamma(a(n,i)/

2)*(1+(fom(i)-mu(n,i))^2/a(n,i))^((a(n,i)+1)/2)); if ipl(n,i) == ’y’ xsmin(n) = max(lower(n,i),plotrange(i,1,n)); xsmax(n) = min(upper(n,i),plotrange(i,2,n)); sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i); xs(n,:) = xsmin(n):sstep(n):xsmax(n); ys(n,:) = subs(func(i),fom(i),xs(n,:)); figure plot(xs(n,:),ys(n,:)) xlabel(name(i)); axis tight end end end clear xsmin xsmax sstep xs ys; F = int(func(1)*fcorr(n),fom(1)); FS = subs(F,fom(1),upper(n,1))-subs(F,fom(1),lower(n,1)); for i = 2:size(distribution,2) F = int(FS*func(i),fom(i)); FS = subs(F,fom(i),upper(n,i))-subs(F,fom(i),lower(n,i)); end Ftot = double(FS); Fp = int(func(1)*fcorr(n),fom(1)); FpS=subs(Fp,fom(1),min(upper(n,1),maxvalue(1)))-subs(Fp,fom(1),

max(lower(n,1),minvalue(1))); if min(upper(n,1),maxvalue(1)) < max(lower(n,1),minvalue(1)) FpS = 0; end for i = 2:size(distribution,2) Fp = int(FpS*func(i),fom(i)); FpS = subs(Fp,fom(i),min(upper(n,i),maxvalue(i)))-

subs(Fp,fom(i),max(lower(n,i),minvalue(i))); if min(upper(n,i),maxvalue(i)) < max(lower(n,i),minvalue(i))


FpS = 0; end end POS(n) = double(FpS/Ftot); if probplot2D == ’y’ JPMprob elseif probplot3D == ’y’ JPMprob endendPOScd plot.filesif probplot2D == ’y’ contourplotendif probplot3D == ’y’ surfplotendif cumplot == ’y’ cumprobplotendcd ..


JPMprob.m

% JPMprob.m %for m = 1:length(crplots(:,1)) plotx = crplots(m,1); ploty = crplots(m,2); xmin(n,plotx) = plotrange(plotx,1,n); xmax(n,plotx) = plotrange(plotx,2,n); xmin(n,ploty) = plotrange(ploty,1,n); xmax(n,ploty) = plotrange(ploty,2,n); aal = 1:nfom; bll = find(aal ~= plotx); cll = find(bll ~= ploty); cond = bll(cll); xmin(n,cond) = condrange(1,cond); xmax(n,cond) = condrange(2,cond); step(n,:) = (xmax(n,:)-xmin(n,:))./nstep(n,:); x(n,:) = xmin(n,plotx):step(n,plotx):xmax(n,plotx); y(n,:) = xmin(n,ploty):step(n,ploty):xmax(n,ploty); [xp,yp] = meshgrid(x(n,:),y(n,:)); invalidxl = find(x(n,:)<lower(n,plotx)); xp(:,invalidxl) = lower(n,plotx); invalidxu = find(x(n,:)>upper(n,plotx)); xp(:,invalidxu) = upper(n,plotx); invalidyl = find(y(n,:)<lower(n,ploty)); yp(invalidyl,:) = lower(n,ploty); invalidyu = find(y(n,:)>upper(n,ploty)); yp(invalidyu,:) = upper(n,ploty); if length(fom) > 2 Fcond = int(func(cond(1))*fcorr(n),fom(cond(1))); FScond = subs(Fcond,fom(cond(1)),min(condrange(2,cond(1)),

upper(n,cond(1))))-subs(Fcond,fom(cond(1)),max(condrange(1,cond(1)),lower(n,cond(1))));

if length(cond) > 1 for i = 2:length(cond) Fcond = int(FScond*func(cond(i)),fom(cond(i))); FScond = subs(Fcond,fom(cond(i)),min(condrange(2,cond(i)),

upper(n,cond(i))))-subs(Fcond,fom(cond(i)),max(condrange(1,cond(i)), lower(n,cond(i))));

end end else FScond = fcorr(n); end ff = FScond*func(plotx)*func(ploty)/Ftot; ff = simplify(ff); ffpro(:,:,n) = subs(ff,fom(plotx),xp); for i = 1:length(y(n,:)) ffprob(i,:,n) = subs(ffpro(i,:,n),fom(ploty),yp(i,1)); end prob(:,:,n) = double(ffprob(:,:,n));end


inputs.m (Empirical Distribution Function)

% inputs.m %method = 2;maxvalue = [90 1000 1000];minvalue = [0 100 5];nstep = [100 100 100 100 100 100 100 100 100 100 100 100];l = [10000 10000 10000 10000];crplots = [1 2 ];condrange = [-100 -100 100 100];probplot2D = ’y’;probplot3D = ’y’;cumplot = ’y’;cp = (0.00005:0.0003:0.05);cc = [0,0.1,.2,.3,.4,.5,.6,.7,.8,.9];aalow = [50 40 ];aahigh = [250 340];name(1) = ’TOC(M$/yr)’;name(2) = ’REV(M$/yr)’;contourcolors = [0.4 0.4 0.4 0.6 0.6 0.6 0.8 0.8 0.8 0.0 0.0 0.0];


EDF.m

% EDF.m %dat = NaN * ones(nfom,max(l),points);fid(1) = fopen(’Data1’,’r’);fid(2) = fopen(’Data2’,’r’);fid(3) = fopen(’Data3’,’r’);fid(4) = fopen(’Data4’,’r’);fid(5) = fopen(’Data5’,’r’);fid(6) = fopen(’Data6’,’r’);fid(7) = fopen(’Data7’,’r’);fid(8) = fopen(’Data8’,’r’);fid(9) = fopen(’Data9’,’r’);fid(10) = fopen(’Data10’,’r’);for n = 1:points dat(:,:,n) = fscanf(fid(n),’%g’,[nfom l(n)]); data(:,:,n) = dat(:,:,n)’;endstatus = fclose(’all’);for n = 1:points xmin(n,:) = min(data(:,:,n),[],1); xmax(n,:) = max(data(:,:,n),[],1); lower(n,:) = max(xmin(n,:),minvalue); upper(n,:) = min(xmax(n,:),maxvalue); fidd = fopen(’passEDF’,’w’); if n == 1 !copy Data1 Data elseif n == 2 !copy Data2 Data elseif n == 3 !copy Data3 Data elseif n == 4 !copy Data4 Data elseif n == 5 !copy Data5 Data elseif n == 6 !copy Data6 Data elseif n == 7 !copy Data7 Data elseif n == 8 !copy Data8 Data elseif n == 9 !copy Data9 Data elseif n == 10 !copy Data10 Data end if nfom == 2 step(n,:) = (xmax(n,:)-xmin(n,:))./nstep(n,:); fprintf(fidd,’%g\n’,l(n),xmin(n,:),xmax(n,:),step(n,:),

lower(n,:),upper(n,:)); status = fclose(fidd); !EDF2.exe x(n,:) = xmin(n,1) + step(n,1)/2:step(n,1):xmax(n,1); y(n,:) = xmin(n,2) + step(n,2)/2:step(n,2):xmax(n,2); fidd = fopen(’passback’,’r’);


i = fscanf(fidd,’%g’,[1 1]); j = fscanf(fidd,’%g’,[1 1]); count = fscanf(fidd,’%g’,[i j]); C(n) = fscanf(fidd,’%g’,[1 1]); countv(n,:) = fscanf(fidd,’%g’,[1 nfom]); status = fclose(fidd); prob(:,:,n) = count(2:i,2:j)/l(n); prob(1,:,n) = (count(1,2:j) + count(2,2:j))/l(n); prob(:,1,n) = (count(2:i,1) + count(2:i,2))/l(n); plotx = 1; ploty = 2; if n == points cd plot.files if probplot2D == ’y’ contourplot end if probplot3D == ’y’ surfplot end if cumplot == ’y’ cumprobplot end cd .. end elseif nfom == 3 fprintf(fidd,’%g\n’,l(n),lower(n,:),upper(n,:)); status = fclose(fidd); !EDF3.exe fidd = fopen(’passback’,’r’); C(n) = fscanf(fidd,’%g’,[1 1]); countv(n,:) = fscanf(fidd,’%g’,[1 nfom]); status = fclose(fidd); elseif nfom == 4 fprintf(fidd,’%g\n’,l(n),lower(n,:),upper(n,:)); status = fclose(fidd); !EDF4.exe fidd = fopen(’passback’,’r’); C(n) = fscanf(fidd,’%g’,[1 1]); countv(n,:) = fscanf(fidd,’%g’,[1 nfom]); status = fclose(fidd); elseif nfom == 5 fprintf(fidd,’%g\n’,l(n),lower(n,:),upper(n,:)); status = fclose(fidd); !EDF5.exe fidd = fopen(’passback’,’r’); C(n) = fscanf(fidd,’%g’,[1 1]); countv(n,:) = fscanf(fidd,’%g’,[1 nfom]); status = fclose(fidd); endendcumulative = C’./l;probtable = zeros(points+1,nfom+2);probtable(2:points+1,1) = transpose(1:points);probtable(1,2:nfom+2) = (0:nfom);probtable(2:points+1,2) = cumulative;


for n = 1:points probtable(n+1,3:nfom+2) = countv(n,:)/l(n);endprobtable


EDF2.exe

program EDF2 implicit none real x, y, xmin, xmax, ymin, ymax, stepx real minx, maxx, miny, maxy, data(15000,2) real count(1001,1001), stepy, epsx, epsy integer k, i, j, l, s, t, C, CX, CY C = 0 CX = 0 CY = 0 open (8, FILE = ’passEDF’, status = ’old’) read (8, *) l, minx, miny, maxx, maxy, stepx read (8, *) stepy, xmin, ymin, xmax, ymax open (9, FILE = ’Data’, status = ’old’) epsx = stepx/100. epsy = stepy/100. do 10 k = 1,l read (9, *) data(k,1), data(k,2) j = 0 do 20 x = minx,maxx+epsx,stepx i = 0 j = j + 1 do 30 y = miny,maxy+epsy,stepy i = i + 1 if (data(k,1) .gt. x - stepx) then if (data(k,1) .le. x) then if (data(k,2) .gt. y - stepy) then if (data(k,2) .le. y) then count(i,j) = count(i,j) + 1 endif endif endif endif30 continue20 continue if (data(k,1) .ge. xmin) then if (data(k,1) .le. xmax) then if (data(k,2) .ge. ymin) then if (data(k,2) .le. ymax) then C = C + 1 endif endif endif endif if (data(k,1) .ge. xmin) then if (data(k,1) .le. xmax) then CX = CX + 1 endif endif if (data(k,2) .ge. ymin) then if (data(k,2) .le. ymax) then CY = CY + 1 endif endif


10 continue open (10, FILE = ’passback’) write (10, *) i, j do 40 s = 1,j do 50 t = 1,i write (10, *) count(t,s)50 continue40 continue write (10, *) C write (10, *) CX write (10, *) CY endfile (10) close (10) close (9) close(8) end


EDF5.exe

program EDF5 implicit none real min(5), max(5), data(15000,5) integer k, l, s, D, C(5) D = 0 do 5 s =1,5 C(s) = 05 continue open (8, FILE = ’passEDF’, status = ’old’) read (8, *) l, (min(s), s=1,5), (max(s), s=1,5) open (9, FILE = ’Data’, status = ’old’) do 10 k = 1,l read (9, *) (data(k,s), s=1,5) if (data(k,1).ge.min(1) .and. data(k,1).le.max(1)) then if (data(k,2).ge.min(2) .and. data(k,2).le.max(2)) then if (data(k,3).ge.min(3) .and. data(k,3).le.max(3)) then if (data(k,4).ge.min(4) .and. data(k,4).le.max(4)) then if (data(k,5).ge.min(5) .and. data(k,5).le.max(5)) then D=D+1 endif endif endif endif endif do 20 s = 1,5 if(data(k,s).ge.min(s) .and. data(k,s).le.max(s))then C(s)=C(s)+1 endif20 continue10 continue open (10, FILE = ’passbackEDF’) write (10, *) D do 40 s = 1,5 write (10, *) C(s)40 continue endfile (10) close (10) close (9) close(8) end


contourplot.m

% contourplot.m %figurehold[xm,ym] = meshgrid(max(min(min(x)),minvalue(plotx))+step(1,plotx)

/2:step(1,plotx):min(max(max(x)),maxvalue(plotx))-step(1,plotx)/2,max(min(min(y))+step(1,ploty)/2,minvalue(ploty)):step(1,ploty):min(max(max(y)),maxvalue(ploty))-step(1,ploty)/2);

plot(xm,ym,’g.’);for n = 1:points [Cont,H] = contour(x(n,:),y(n,:),prob(:,:,n),cp,’-’); set(H,’Color’,contourcolors(n,:))endycp = min(min(x)):step(1,plotx):max(max(x));xcp = min(min(y)):step(1,ploty):max(max(y));if minvalue(plotx) > min(min(x)) lo = ones(1,length(xcp))*minvalue(plotx); plot(lo,xcp) clear loendif minvalue(ploty) > min(min(y)) lo = ones(1,length(ycp))*minvalue(ploty); plot(ycp,lo) clear loendif maxvalue(plotx) < max(max(x)) lo = ones(1,length(xcp))*maxvalue(plotx); plot(lo,xcp) clear loendif maxvalue(ploty) < max(max(y)) lo = ones(1,length(ycp))*maxvalue(ploty); plot(ycp,lo) clear loendif sum(aalow) == 0 axis tightelse axis([aalow(plotx) aahigh(plotx) aalow(ploty) aahigh(ploty)])endxlabel(name(plotx));ylabel(name(ploty));print -djpeg95 prob2D.jpghold


surfplot.m

% surfplot.m %figureif sp == 0 for n = 1:points if method == 2 tprob(:,:,n) = fliplr(prob(:,:,n)); graph = bar3(y(n,:),tprob(:,:,n)); view(155,20); else graph = surf(x(n,:),y(n,:),prob(:,:,n)); view(-15,30); end set(graph,’EdgeColor’,[0.6 0.6 0.6]); hold on endelse if method == 2 tprob = fliplr(prob(:,:,sp)); graph = bar3(y(sp,:),tprob); view(155,20); else graph = surf(x(sp,:),y(sp,:),prob(:,:,sp)); view(-15,30); end set(graph,’EdgeColor’,[0.6 0.6 0.6]); hold onendaxis tightxlabel(name(plotx));ylabel(name(ploty));zlabel(’Density’);print -djpeg95 prob3D.jpg


cumprobplot.m

% cumprobplot.m %if method == 2 step = ones(points,nfom);end[ey,ex] = size(prob(:,:,1));for n = 1:points

for i = 1:round(t):eykt = 1+(i-1)/round(t);for j = 1:round(t):ex

lt = 1+(j-1)/round(t);if direction(plotx) == ’l’

if direction(ploty) == ’l’cmp(kt,lt,n) = sum(sum(prob(i:ey,j:ex,n)));

elseif direction(ploty) == ’s’cmp(kt,lt,n) = sum(sum(prob(1:i,j:ex,n)));

endelseif direction(plotx) == ’s’

if direction(ploty) == ’l’cmp(kt,lt,n) = sum(sum(prob(i:ey,1:j,n)));

elseif direction(ploty) == ’s’cmp(kt,lt,n) = sum(sum(prob(1:i,1:j,n)));

endend

endendcump(:,:,n) = cmp(:,:,n)*prod(step(n,:));

endfigurefor n = 1:points

[Cont,H] = contour(x(n,:),y(n,:),cump(:,:,n),cc,’-’);set(H,’Color’,contourcolors(n,:))clabel(Cont,H,’fontsize’,8,’color’,contourcolors(n,:),’labelsp

acing’,1000);hold on

endif maxvalue(plotx) > xmax(n,plotx) cx = ones(1,length(y(n,:)))*minvalue(plotx); if maxvalue(ploty) > xmax(n,ploty) cy = ones(1,length(x(n,:)))*minvalue(ploty); elseif minvalue(ploty) < xmin(n,ploty) cy = ones(1,length(x(n,:)))*maxvalue(ploty); endelseif minvalue(plotx) < xmin(n,plotx) cx = ones(1,length(y(n,:)))*maxvalue(plotx); if maxvalue(ploty) > xmax(n,ploty) cy = ones(1,length(x(n,:)))*minvalue(ploty); elseif minvalue(ploty) < xmin(n,ploty) cy = ones(1,length(x(n,:)))*maxvalue(ploty); endendplot(x,cy)plot(cx,y)if sum(aalow) == 0


axis tightelseaxis([aalow(plotx) aahigh(plotx) aalow(ploty) aahigh(ploty)])endxlabel(name(plotx)); ylabel(name(ploty)); print -djpeg95 cum2D.jpg

Oliver Bandte Bibliography 209

BIBLIOGRAPHY

Airbus Industries, “Products – Technical - Specifications” <http://www.airbus.com>

August, 2000.

Ang, A. H.-S., Tang, W. H., Probability Concepts in Engineering Planning and Design,

Volume I, John Wiley & Sons, Inc., New York, 1984.

Asimow, M., Introduction to Design, Prentice-Hall, Inc., Englewood Cliffs, NJ, 1962.

Beakley, G.C., Leach H.W., Engineering - An Introduction to a Creative Profession, 2nd

Edition, The Macmillan Company, New York, 1972.

Blanchard, B.S., Systems Engineering Management, 2nd Edition, John Wiley & Sons,

Inc., New York, 1998.

Blanchard, B.S., Fabrycky, W.J., Systems Engineering and Analysis, 3rd Edition,

Prentice-Hall, New York, 1998.

Boeing Company, The, “Commercial Aircraft Product Information”

<http://www.boeing.com/commercial/> August, 2000.

Box, G.E.P., Draper, N.R., Empirical Model-Building and Response Surfaces, John

Wiley & Sons, Inc., New York, 1987.


Box, G.E.P., Hunter, W.G., Hunter, J.S., Statistics for Experimenters, John Wiley &

Sons, Inc., New York, 1978.

Branch, M.A., Grace, A., MATLAB® - Optimization Toolbox (User’s Guide), The

MathWorks, Inc., 1996.

Chen, W., et al., Integration of Response Surface Method with the Compromise Decision

Support Problem in Developing a General Robust Design Procedure, Advances in

Design Automation (Azarm, S., et al. Eds.), Vol. 82-2, ASME, New York, 1995.

Cheng, B., Titterington, D.M., Neural Networks: A Review from a Statistical Perspective,

Statistical Science, Vol. 9, No. 1, 2-54, 1994.

Cleland, D.I., King, W.R., Systems Analysis and Project Management, McGraw-Hill,

Inc., New York, 1983.

DeLaurentis, D.A., A Stochastic Approach to Aircraft Design Incorporating Guidance

and Control Uncertainties, Ph.D. Thesis, Georgia Institute of Technology, 1998.

DeLaurentis, D.A., Calise, A., Schrage, D.P., Mavris, D.N., Reduced Order Guidance

Methods and Probabilistic Techniques in Addressing Mission Uncertainty, 6th

AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and

Optimization, Bellevue, WA, September, 1996, AIAA-96-4174.


DeLaurentis, D.A., Mavris, D.N., Schrage, D.P., System Synthesis in Preliminary Aircraft

Design Using Statistical Methods, 20th Congress of the International Council on the

Aeronautical Sciences, Sorrento, Italy, September, 1996.

Dieter, G.E., Engineering Design, 2nd Edition, McGraw-Hill, Inc., New York, 1991.

Dixon, J.R., Design Engineering: Inventiveness, Analysis, and Decision Making,

McGraw-Hill Book Company, New York, 1966.

Ertas, A., Jones, J.C., The Engineering Design Process, John Wiley & Sons, New York,

1993.

Fleming, P.J., Application of Multiobjective Optimization to Compensatory Design for

SISO Control Systems, Electronics Letters, Vol.22, No.5, pp.258-259, 1986.

Fox, E.P., The Pratt & Whitney Probabilistic Design System, 35th AIAA/ASME/ASCE/

AHS/ASC Structures, Structural Dynamics, and Materials Conference, Hilton Head,

SC, April, 1994, AIAA-94-1442.

Fox, E.P., Reh, S, On the Accuracy of Various Probabilistic Methods, 41st

AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials

Conference, Atlanta, April 2000, AIAA-2000-1631.

Galloway, T.L., Mavris, D.N., Aircraft Life Cycle Cost Analysis (ALCCA) Program,

NASA Ames Research Center, September 1993.


Garvey, P.R., Probability Methods for Cost Uncertainty Analysis – A Systems Engineers

Perspective, Marcel Dekker, Inc., New York, Basel, Hongkong, 1999.

Garvey, P.R., Taub, A.E., Joint Probability Model for Cost and Schedule Uncertainties,

26th Annual Department of Defense Cost Analysis Symposium, September, 1992.

Gembicki, F.W., Performance and Sensitivity Optimization: A Vector Index Approach,

Ph.D. Thesis, Case Western Research University, 1974.

Giunta, A.A., et al., Wing Design for a High-Speed Civil Transport Using a Design of

Experiments Methodology, 6th AIAA/ USAF/NASA/ISSMO Symposium on

Multidisciplinary Analysis and Optimization, Bellevue, WA, September, 1996,

AIAA-96-4001.

Hazelrigg, G.A., Systems Engineering: an Approach to Information-Based Design,

Prentice-Hall, Inc., Upper Saddle River, NJ, 1996.

Hogg, R.V., Tanis, E.A., Probability and Statistical Inference, Macmillan Publishing

Company, New York, 1993.

Huber, K.-P., Berthold, M.R., Szczerbicka, H., Analysis of Simulation Models with Fuzzy

Graph based Metamodeling, Performance Evaluation, Vol. 27&28, North-Holland,

October 1996.


Hwang, C.-L., Masud, A.S.Md., Multiple Objective Decision Making – Methods and

Applications (Lecture Notes in Economics and Mathematical Systems; 164), Springer

Verlag, Berlin, Heidelberg, New York, 1979.

Hwang, C.-L., Yoon, K., Multiple Attribute Decision-Making – Methods and

Applications (Lecture Notes in Economics and Mathematical Systems; 186), Springer


Jones, B., MATLAB® - Statistics Toolbox (User’s Guide), The MathWorks, Inc., 1997.

Kaufman, M., et al., Variable-Complexity Response Surface Approximations for Wing

Structural Weight in HSCT Design, 34th Aerospace Sciences Meeting & Exhibit,

Reno, NV, January, 1996, AIAA-96-0089.

Khalessi, M.R., Lin, H.-Z., Most-Probable-Point-Locus Structural Reliability Method,

Proceedings of 34th Structures, Structural Dynamics, and Materials Conference, April,

1993, AIAA-93-1439.

Kleijnen, J.P.C., Statistical Techniques in Simulation, Part I, Marcel Dekker, Inc., New

York, 1974.

Kleijnen, J.P.C., Statistical Tools for Simulation Practitioners, Marcel Dekker, Inc., New

York, 1987.


Mavris, D.N., Bandte, O., Effect of Mission Requirements on the Economic Robustness of

an HSCT, 18th Annual Conference of the International Society of Parametric

Analysts, Cannes, France, June, 1996.

Mavris, D.N., Bandte, O., A Probabilistic Approach to Multivariate Constrained Robust

Design Simulation, World Aviation Congress and Exposition, Anaheim, CA, October,

1997, SAE-97-5508.

Mavris, D.N., Bandte, O., Comparison of Two Probabilistic Techniques for the

Assessment of Economic Uncertainty, Joint Cost Management Proceedings, AACE

International, 1998.

Mavris, D.N., Bandte, O., DeLaurentis, D.A., Robust Design Simulation: A Probabilistic

Approach to Multidisciplinary Design, Journal of Aircraft, Vol. 36, No. 1, January –

February, 1999, pp. 298-307

Mavris, D.N., Bandte, O., Schrage, D.P., Economic Uncertainty Assessment of an HSCT

Using a Combined Design of Experiments/ Monte-Carlo Simulation Approach, 17th

Annual Conference of the International Society of Parametric Analysts, San Diego,

CA, June, 1995.

Mavris, D.N., Bandte, O., Schrage, D., Application of Probabilistic Methods for the

Determination of an Economically Robust HSCT Configuration, AIAA/USAF/

NASA/ISSMO Multidisciplinary Analysis and Optimization Conference, Bellevue,

WA, September, 1996, AIAA-96-4090.


Mavris, D.N., DeLaurentis, D.A., A Stochastic Design Approach for Aircraft

Affordability, 21st Congress of the International Council on the Aeronautical Sciences,

Melbourne, Australia, September, 1998, ICAS-98-6.1.3.

Mavris, D.N., Kirby, M.R., Forecasting the Impact of Technology Infusion on Subsonic

Transport Affordability, World Aviation Congress and Exposition, September, 1998,

SAE-98-5576.

Mavris, D.N., Kirby, M.R., Qiu, S., Technology Impact Forecasting for a High Speed

Civil Transport, World Aviation Congress and Exposition, September, 1998,

SAE-985547.

McCullers, L.A., Flight Optimization System, Computer Program and Users Guide,

Version 5.7, NASA Langley Research Center, Hampton, VA, December 1994.

Nas, T.F., Cost Benefit Analysis Theory and Application, SAGE Publications, Thousand

Oaks, CA, 1996.

Ostrofsky, B., Design, Planning, and Development Methodology, Prentice-Hall,

Englewood Cliffs, NJ, 1977.

Osyczka, A., Multicriterion Optimization in Engineering with FORTRAN Programs,

Ellis Horwood Limited, Chichester, 1984.

Oxford Atlas of the World, 7th Edition, Oxford University Press, New York, 1999.


Raymer, D.P., Aircraft Design: a Conceptual Approach, 2nd Edition, American Institute

of Aeronautics and Astronautics, Inc., Washington, DC, 1992.

Reklaitis, G.V., Ravindran, A., Ragsdell, K.M., Engineering Optimization – Methods and

Applications, John Wiley & Sons, Inc., New York, 1983.

Robinson, D.G., CRAX/Cassandra User Manual, Sandia Report SAND99-3162, Sandia

National Laboratories, 1999.

Ross, S.M., Introduction to Probability Models, Academic Press, Inc., San Diego, CA,

1993.

SAE, G-11 Probabilistic Methods Committee, Perceptions and Limitations Inhibiting the

Application of Probabilistic Methods, SAE Standard AIR5086, SAE, Warrendale,

1998.

Saaty, T.L., The Analytic Hierarchy Process: Planning, Priority Setting, Resource

Allocation, McGraw-Hill, New York, 1980.

Sacks, J., Schiller, S.B., Welch, W.J., Design for Computer Experiments, Technometrics,

Vol. 31, No. 1, February, 1989.

Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P., Design and Analysis of Computer

Experiments, Statistical Science, Vol. 4, No. 4, February, 1989.

Sakata, I.F., Davis, G.W., Evaluation of Structural Design Concepts for Arrow-Wing

Supersonic Cruise Aircraft, NASA CR-2667, May, 1977.


Sen, P., Yang, J.-B., Multiple Criteria Decision Support in Engineering Design, Springer

Verlag, Berlin, Heidelberg, New York, London, 1998

Shooman, M.L., Probabilistic Reliability; An Engineering Approach, McGraw-Hill, New

York, 1968.

Siddall, J.N., Optimal Engineering Design, Marcel Dekker, Inc., New York and Basel,

1982.

Silver, G.A., Silver, J.B., Introduction to Systems Analysis, Prentice-Hall, Inc.,

Englewood Cliffs, NJ, 1976.

Southwest Research Institute, FPI User’s and Theoretical Manual, San Antonio, TX,

1995.

Stadler, W., Caveats and Boons of Multicriteria Optimization, Microcomputers in Civil

Engineering, Vol. 10, Blackwell, Cambridge, MA, 1995, pp. 291-299.

Stadler, W., Dauer, J., Multicriteria Optimization in Engineering: A Tutorial and Survey,

Structural Optimization: Status and Promise (ed. Kamat, M.P.), Vol 150, Progress in

Astronautics and Aeronautics, AIAA, Washington, DC, 1992, pp. 209-244.

Starr, M.K., Product Design and Decision Theory, Prentice Hall, New York, 1963.

Steuer, R.E., Multiple Criteria Optimization: Theory, Computation, and Application,

John Wiley & Sons, New York, 1986.


Stolarski, T.A., Analytical Design Techniques, Elsevier Science BV, Amsterdam, 1995.

Strang, G., Introduction to Linear Algebra, Wellesley-Cambridge Press, Wellesley, MA,

1993.

Tai, J.C.M., A Multidisciplinary Design Approach to Size Stopped Rotor/Wing

Configurations Using Reaction Drive and Circulation Control, Ph.D. Thesis, Georgia

Institute of Technology, 1998.

Tong, Y.L., The Multivariate Normal Distribution, Springer Series in Statistics, Springer


Unal, R., Stanley, D.O., Joyner, C.R., Parameter Model Building and Design

Optimization Using Response Surface Methods, Journal of Parametrics, Washington,

DC, May, 1994.

Vanderplaats, G.N., Numerical Optimization Techniques for Engineering Design, 3rd

Edition, Vanderplaats, Research & Development, Inc., 1999.

Webster’s New World Dictionary of American English, 3rd College Edition, Prentice

Hall, New York, 1996.

Welch, W.J., et al., Screening, Predicting, and Computer Experiments, Technometrics,

Vol. 34, No. 1, February, 1992.


Wu, Y.-T., Burnside, O.H., Dominguez, J., Efficient Probabilistic Fracture Mechanics

Analysis, 4th International Conference on Numerical Methods in Fracture Mechanics,

Pineridge Press, Swansea, U.K., May, 1987.

Wu, Y.-T., Millwater, H.R., Cruse, T.A., Advanced Probabilistic Structural Analysis

Method for Implicit Performance Functions, AIAA Journal, Vol.28, No.3, September

1990, pp.1663-1669.

Zeleny, M., Multiple Criteria Decision Making, McGraw-Hill, Inc., New York, 1982.

Oliver Bandte Vita 220

VITA

Oliver Bandte was born in Hamburg, Germany on January 24th, 1970. He lived in

Hamburg till after graduation from Gymnasium Meiendorf (high school) in 1989. He

subsequently moved to Braunschweig, Germany, where he completed a degree in

mechanical engineering (Diplom-Ingenieur) received from the Technische Universität

Braunschweig in 1994. Since then he earned a Master of Science degree (no designation)

from the Georgia Institute of Technology (1996).

aircraft design thesis

Documents