the foundations of decision analysis...

Advances: Foundations of DA Revisited of 44 Ch 03 060520 V10

3 The Foundations of Decision Analysis Revisited

Ronald A. Howard

ABSTRACT

For centuries people have speculated on how to improve decision-making without much

professional help in developing clarity of action. Over the last several decades several

important supporting fields have been integrated to provide a discipline, decision

analysis, which can aid decision-makers in all fields of endeavor: business, engineering,

medicine, law, and personal life. Since uncertainty is the most important feature to

consider in making decisions, the ability to represent knowledge in terms of probability,

to see how to combine this knowledge with preferences in a reasoned way, to treat very

large and complex decision problems using modern computation, and to avoid common

errors of thought have combined to produce insights heretofore unobtainable. The

limitation in practice lies in our willingness to use reason rather than in any shortcoming

of the field. This chapter discusses the sources of the discipline, the qualities desired in a

decision process, the logic for finding the best course of action, the process of focusing

attention on important issues in attaining clarity of action, the need for clear and powerful

distinctions to guide our thinking (since most decisions derive from thought and

conversation rather than from computation), and the challenges to the growth of the

discipline.


CONTENTS

Origins

The Motivating Application

Pillars of Decision Analysis

The First Pillar: Systems Analysis

The Second Pillar: Decision Theory

The Third Pillar: Epistemic Probability

The Fourth Pillar: Cognitive Psychology

Edifice of Decision Analysis

Decision Quality

Group Decision Making

Clairvoyance

Desiderata

The Rules

Rule 1: The Probability Rule

Rule 2: The Order Rule

Rule 3: The Equivalence Rule

Rule 4: The Substitution Rule

Rule 5: The Choice Rule

Practice of Decision Analysis

Decision Engineering

Influence, Relevance, and Decision Diagrams

Options


Cogency versus Verisimilitude

Ethics

Language of Decision Analysis

Confusions

Uncertainty about Probability

Deal Characterization and Deal Desirability

Challenges of Decision Analysis

Classical Statistics Persists

Dealing with Multiple Attributes

Direct and Indirect Values

Treating Multi-Attribute Problems Using a Value Function

Other Approaches to Multi-Attribute Problems

Weight and Rate

Analytic Hierarchy Process

Risk Preference

Life and Death Decisions

Future of Decision Analysis

Epilog

Acknowledgment


Revisiting the foundations of decision analysis means seeing what those foundations have

been, how they have evolved, and how well they will serve in the future. Since the entire

book is generally concerned with this subject, this chapter will be a personal view of the

development of decision analysis. Other chapters will discuss many of the topics

commented on here in greater depth. Certain of my opinions may be idiosyncratic, as you

will see as you read further.

I will briefly repeat here comments from original writings: they should be consulted for a

deeper understanding.

Origins

When I was asked to do this chapter, I thought of it as an opportunity to review the progress of

the field since its inception. I decided to return to my 1965 paper entitled "Decision Analysis:

Applied Decision Theory" (DAADT) (Howard, 1966) wherein I define the term "decision

analysis" for the first time:

The purpose of this article is to outline a formal procedure for the analysis of decision

problems, a procedure that I call “decision analysis.” We shall also discuss several of the

practical problems that arise when we attempt to apply the decision analysis formalism.

By the way, I decided to call the field “decision analysis” rather than “decision engineering”

because the latter term sounded manipulative, though in fact it is more descriptive.

The following paragraph from DAADT provides a definition of decision analysis:

Decision analysis is a logical procedure for the balancing of the factors that influence a

decision. The procedure incorporates uncertainties, values, and preferences in a basic


structure that models the decision. Typically, it includes technical, marketing,

competitive, and environmental factors. The essence of the procedure is the construction

of a structural model of the decision in a form suitable for computation and manipulation;

the realization of this model is often a set of computer programs.

Not having read the paper for some time, and expecting it to be antiquated, I was pleased to

see how relevant some of the comments are today. For example, here is the discussion of the

most fundamental distinction underlying decision analysis:

Having defined a decision, let us clarify the concept by drawing a necessary distinction

between a good decision and a good outcome. A good decision is a logical decision --

one based on the uncertainties, values, and preferences of the decision maker. A good

outcome is one that is profitable or otherwise highly valued. In short, a good outcome is

one that we wish would happen. Hopefully, by making good decisions in all the situations

that face us we shall ensure as high a percentage as possible of good outcomes. We may

be disappointed to find that a good decision has produced a bad outcome or dismayed to

learn that someone who has made what we consider to be a bad decision has enjoyed a

good outcome. Yet, pending the invention of the true clairvoyant, we find no better

alternative in the pursuit of good outcomes than to make good decisions.

The distinction between decision and outcome is still not clear for most people. When

someone makes an investment, and then loses money, he often says that he made a bad decision.

If he would make the same decision again if he did not know how it would turn out, then he

would be more accurate in saying that he made a good decision and had a bad outcome. Jaynes

traces this distinction back to Herodotus (Jaynes, 1986):

From the earliest times this process of plausible reasoning preceding decisions has been

recognized. Herodotus, in about 500 BC, discusses the policy decisions of the Persian kings. He


notes that a decision was wise, even though it led to disastrous consequences, if the evidence at

hand indicated it as the best one to make; and that a decision was foolish, even though it led to

the happiest possible consequences, if it was unreasonable to expect those consequences.

Let me now turn to what motivated the writing of the paper.

The Motivating Application

I taught in electrical engineering and industrial management at MIT from 1958 through 1964.

One of my mentors at MIT, Bill Linvill, had taken a position at Stanford University. He invited

me to visit him there for the ’64-’65 academic year. While I was at MIT, I taught Statistical

Decision Theory and Markov decision processes in the General Electric Modern Engineering

Course, which they gave to middle engineering management. When one of the participants

learned I was going out to Stanford, he asked if I could teach the same course in San Jose to

employees of the GE nuclear power division, and I agreed.

At the end of the third weekly lecture in decision theory presented in San Jose, one of the

engineers said that the division was facing a major decision problem with both technical and

business implications. The question was whether to put a superheater of steam on their nuclear

power reactors. He asked whether what we were discussing in class could help with this

problem.

I replied, “Why not?” We spent eight months working on how to put together the dynamic,

preference, and uncertainty issues they faced, marking the beginning of decision analysis and

providing the material that motivated the first decision analysis paper quoted above.


Pillars of Decision Analysis

It is certainly true that we stand on the shoulders of giants. I used to think that there were two

pillars supporting decision analysis, but I came to realize that there were three, and then four. I

shall briefly discuss the first two pillars and emphasize the last two.

The First Pillar: Systems Analysis

Bill Linvill introduced me to systems analysis. Systems analysis grew out of World War II and

was concerned with understanding dynamic systems. Key notions were those of state variables,

feedback, stability, and sensitivity analysis. The field of systems engineering is currently in a

state of resurgence. Decision analysis and systems engineering have many complementary

features (Howard, 1965, 1973).

The Second Pillar: Decision Theory

Decision theory is concerned primarily with making decisions in the face of uncertainty. Its roots

go back to Daniel Bernoulli (Bernoulli, 1738) and Laplace. Bernoulli introduced the idea of

logarithmic utility to explain the puzzle called the St. Petersburg paradox. In the most influential

book on probability ever written (Laplace, 1812), Laplace discusses the esperance mathematique

and the esperance morale. Today we would call these the mean and the certain equivalent.

Howard Raiffa showed how to treat many of the problems of statistics in Bayesian form, and

how to use tree structures to select the best alternative. I learned much from Howard while I

assisted him in teaching the Institute of Basic Mathematics for Application to Business.


The Third Pillar: Epistemic Probability

I did not appreciate the importance of this pillar until well after I began teaching statistical

decision theory. My epiphany began with a manuscript sent to me by Myron Tribus, Dean of

Engineering at Dartmouth. It was a paper of Ed Jaynes (Jaynes, 2003), a professor of physics at

Washington University. Since I needed a haircut, I took it with me to a small barbershop on

Massachusetts Avenue in Cambridge. As I sat in the barber’s chair, I read the first few pages and

thought, “This is pretty silly”. When I arose from the chair 20 minutes later, I had become

completely converted to Jaynes’s way of thinking about probability, and I have been in his debt

ever since.

Jaynes taught that there is no such thing as an objective probability: a probability reflects a

person’s knowledge (or equivalently ignorance) about some uncertain distinction. People think

that probabilities can be found in data, but they cannot. Only a person can assign a probability,

taking into account any data or other knowledge available. Since there is no such thing as an

objective probability, using a term like "subjective probability" only creates confusion.

Probabilities describing uncertainties have no need of adjectives.

This understanding goes back to Cox (2001), Jeffreys (1939), Laplace (1996) and maybe

Bayes, yet somehow it was an idea that had been lost over time. A famous scientist put it best

over 150 years ago:

The actual science of logic is conversant at present only with things either certain,

impossible, or entirely doubtful, none of which (fortunately) we have to reason on.

Therefore the true logic for this world is the calculus of Probabilities, which takes

account of the magnitude of the probability which is, or ought to be, in a reasonable

man's mind. (Maxwell, 1850)

I ask students who have taken a probability class about the origin of the probabilities they use in


the classroom and in their assignments. It turns out that these probabilities originate from the

professor or from a very simple rule based on coin tossing, balls in urns, or card playing that says

to make all elemental probabilities equal. Probability class then teaches you how to transform

this set of probabilities into probabilities of derivative events, like two heads out of ten tosses of

a coin or a royal flush – from inputs to outputs. There is very little discussion of where

probabilities come from, and where they come from has everything to do with the use of

probability. I have never seen an actual decision problem where the assignment of probabilities

could be done using the probability class rule.

DAADT emphasized the importance of the epistemic view:

Another criticism is, “If this is such a good idea, why haven’t I heard of it before?” One

very practical reason is that the operations we conduct in the course of a decision analysis

would be expensive to carry out without using computers. To this extent decision analysis

is a product of our technology. There are other answers, however. One is that the idea of

probability as a state of mind and not of things is only now regaining its proper place in

the world of thought. The opposing heresy lay heavy on the race for the better part of a

century. We should note that most of the operations research performed in World War II

required mathematical and probabilistic concepts that were readily available to Napoleon.

One wonders about how the introduction of formal methods for decision making at that

time might have affected the course of history.

Over the years, many people have tried to modify the probability theory of Bayes, Laplace,

Kolmogorov, and Jaynes for some purpose. Perhaps someday we will see a useful contribution

from such efforts; I believe that, at present, they only serve to make these giants roll over in their

graves.


The Fourth Pillar: Cognitive Psychology

In the 1960s few appreciated the important role that cognitive psychology would play in

understanding human behavior. At the time of DAADT, we just did our best to help experts

assign probabilities. In the 1970s the work of Tversky, Kahneman, and others provided two

valuable contributions. First, it showed that people making decisions relying only on their

intuition were subject to many errors that they would recognize upon reflecting on what they had

done. This emphasized the need for a formal procedure like decision analysis to assist in making

important decisions. The second contribution was to show the necessity for those who are

assisting in the probability and preference assessments to be aware of the many pitfalls that are

characteristic of human thought. Tversky and Kahneman called these heuristics -- methods of

thought that could be useful in general but could trick us in particular settings. We can think of

these as the "optical illusions" of the mind.

An important distinction here is that between "descriptive" and "normative" decision-making.

Descriptive decision-making, as the name implies, is concerned with how people actually make

decisions. The test of descriptive decision-making models is whether they actually describe

human behavior. Normative decision-making is decision-making according to certain rules, or

norms, that we want to follow in our decision-making processes.

To illustrate, I might make mistakes, descriptively, in carrying out operations in arithmetic. I

know they are mistakes because I want to follow the norms of arithmetic. Any violation of the

norms I call a mistake.

We know there is a conflict between our reasoning (normative) and our temptations

(descriptive) that can resolve in favor of either, as illustrated by the person eating a bowl of

peanuts at a cocktail party and saying, "I know I am going to regret this tomorrow."


Many theorists have attempted to change the norms of decision-making to make the norms

agree with descriptive behavior. Every attempt to do so that I have seen creates more problems

than it solves (Howard, 1992a).

Edifice of Decision Analysis

General Concepts

You can think of a decision as a choice among alternatives that will yield uncertain futures, for

which we have preferences. To explain the formal aspects of decision analysis to both students

and to executives I use the image of the three-legged stool shown in Figure 3.1 (Howard, 2000).

Figure 3.1 Decision Essentials


The legs of the stool are the three elements of any decision: what you can do, the alternatives;

what you know, the information you have; and what you want, your preferences. Collectively,

the three legs represent the decision basis, the specification of the decision. Note that if any leg is

missing, there is no decision to be made. If you have only one alternative, then you have no

choice in what you do. If you do not have any information linking what you do to what will

happen in the future, then all alternatives serve equally well because you do not see how your

actions will have any effect. If you have no preferences regarding what will happen as a result of

choosing any alternative, then you will be equally happy choosing any one. The seat of the stool

is the logic that operates on the decision basis to produce the best alternative. We shall soon be

constructing the seat to make sure that it operates correctly.

The stool can be placed anywhere and used to make a decision. However, the most important

choice you make is where to place it. Placement of the stool represents the frame of the decision,

the declaration by the decision maker of what decision is under consideration at this

time. The frame will influence all elements of the decision basis. Framing a decision of where to

live as a renting decision rather than a buying decision will affect the alternatives, information,

and preferences appropriate to the decision basis.

The decision hierarchy of Figure 3.2 shows how the frame separates what is to be decided

upon now from two other potential sources of decisions.


Figure 3.2 The Decision Hierarchy

The top of the hierarchy in the figure represents higher-level decisions with alternatives that

are taken as given at this time. The bottom of the hierarchy represents decisions that will be

made in the future following the decision under consideration. Selection of a proper frame is

perhaps the most important task in decision analysis.

Finally note the person seated on the stool. This figure reminds us that the frame, and every

element of the decision basis, must be a declaration by the decision maker. Decisions are not

found in nature, they are creations of the human mind.

Decision Quality

Decision quality comprises the six elements of the stool. A high quality decision has a proper

frame, a selection of alternatives that respond to the frame, reliable information to an extent

appropriate to the frame, and considered preferences on possible futures. The logic to arrive at a

course of action must be sound, and the decision maker must be committed both to the process


and to the significance of the decision. The decision quality spider shown in Figure 3.3 is a

graphical representation of the qualitative attainment of these elements in any particular decision.

Individuals and groups find it helpful in assessing their decision process.

The distance from the inner circle to the outer one represents the degree of achievement for

each element. The outer circle represents the proper balancing of these elements for this

particular decision. The analysis is not balanced if too many alternatives are considered, too

much information of little value relative to cost is gathered, et cetera. The resulting picture

displays the deficiencies in any of the elements of decision quality.

Appropriate Frame Commitment to Action

CreativeAlternatives

Reliable Information Clear Preferences

0% 100% CorrectLogic

100%: Further improvement uneconomical

including Models

Appropriate Frame Commitment to Action

CreativeAlternatives

Reliable Information Clear Preferences

0% 100% CorrectLogic

100%: Further improvement uneconomical

including Models

Figure 3.3 The Decision Quality Spider

Group Decision Making

Any person, and an organization operating under the direction of a person, can use decision

analysis. An organization using decision analysis is agreeing to act as if it were a single entity

using the logic of a person. Separate groups might have the assignment of creating the frame and

the elements of the decision basis. (Howard, 1975). The analysis must use the same high-quality

logic appropriate for a person soon to be described.


Even when the decision maker is one person, that person may consider the consequences of

the decision on other people. It is useful to define a stakeholder in a decision as "someone who

can affect or will be affected by the decision". Stakeholders can be as disparate as regulators and

customers.

Clairvoyance

A useful construct in achieving clarity in all the dimensions of uncertainty in a decision is that of

the clairvoyant. The clairvoyant can tell us the resolution of any uncertain distinction past,

present, or future as long as 1) the clairvoyant need not exercise any judgment in stating this

resolution and 2) the resolution does not depend upon any future action of the decision maker

unless that action is specified. We say that a distinction meeting these conditions has met the

clarity test. We cannot understand what a distinction like "technical success" means unless it

meets the clarity test. Assigning probabilities to distinctions that do not meet the clarity test is an

exercise in futility.

Once we have the notion of clairvoyance, we can speak of the improvement that we might

make in a decision if clairvoyance on one or more of the uncertainties in the decision were

available. If the prospects of the decision are completely describable by a value measure we can

compute the value of clairvoyance (Howard, 1966a). The most that should be paid for any

information gathering activity or experiment is the value of clairvoyance on the results of that

activity.

As stated in DAADT:

Thus the decision analysis is a vital structure that lets us compare at any time the values

of such alternatives as acting, postponing action and buying information, or refusing to

consider the problem further. We must remember that the analysis is always based on the


current state of knowledge. Overnight there can arrive a piece of information that changes

the nature of the conclusions entirely. Of course, having captured the basic structure of

the problem, we are in an excellent position to incorporate any such information.

Desiderata

In developing any theory, it is useful to specify the desirable properties we would like to have:

desiderata. Here we shall present desiderata for a theory of decision (Howard, 1992a). We define

the decision composite as the axioms supporting the theory plus all of the theorems that follow

from them.

1. The decision composite must allow me to form inferences about uncertain distinctions

even in the absence of a decision problem. This means that probability must stand on its

foundation, in accordance with our discussion of epistemic probability.

2. The decision composite must be applicable to any decision I face regardless of type or

field.

3. The decision composite must require that I be indifferent between two alternatives I

consider to have the same probabilities of the same consequences. In other words, I must

be indifferent between two alternatives for which I have created the same descriptions.

4. Reversing the order of contemplating uncertain distinctions should not change inference

or decision. This means, in particular, that changing the order of receiving a given body

of information, including alternatives, should not change any inference or decision. This

property is sometimes called "invariance to data order".

5. If I prefer alternative 1 over alternative 2 when the uncertain event A occurs and if I

prefer 1 over 2 when A does not occur, then I must prefer alternative 1 over 2 when I am

uncertain about the occurrence of A.


6. Once I have ordered my alternatives from best to worst, the non-informative removal of

any of them does not change the preference order of the remaining alternatives. Non –

informative means the removal of the alternative does not provide new information about

the remaining alternatives.

7. The addition of a non-informative new alternative to the basis cannot change the ranking

of the original alternatives.

8. The ability to obtain free clairvoyance on any uncertainty cannot make the decision

situation less attractive.

9. At this epoch my thoughts about how I will behave and choose in the future must be

consistent.

If the consequences of a decision are completely describable in terms of a value measure

chosen so that more will always be preferred to less, then the desiderata can be further refined.

10. If an alternative has various value consequences with associated probabilities, then I must

be able to compute the amount of the value measure I would have to receive in exchange

for the alternative to be indifferent to following it. This selling price of the alternative I

shall call the certain equivalent. This amount will be negative for undesirable alternatives.

11. I must be able to compute the value added by a new alternative: the value must be

nonnegative.

12. I must be able to compute the value of clairvoyance on any uncertain distinction or

collection of uncertain distinctions; the value of clairvoyance cannot be negative.

13. Payments of value that cannot be changed must have no effect on future decisions.

14. Since I am choosing among uncertain futures, there must be no willingness-to-pay to

avoid regret.


The Rules

How could we construct a systematic logical process to serve as the seat of the stool in Figure

3.1? The answer as that we have to agree in our decision-making to follow a set of rules, or

norms, for our decision process. I consider them the rules of actional thought, thought about

action. These norms -- some would call them axioms -- are the foundation of the decision

composite. I shall state them here as requirements that the decision maker is placing on himself,

rather than as requirements imposed upon him. While every decision made using this normative

process must follow the rules, once they are acknowledged they will seldom be referred to in a

formal analysis. Just as a carpenter relies on the axioms of geometry, perhaps without even

knowing them, the decisions made using decision analysis procedures should be so self-evident

in their correctness that there is rarely a need to mention the rules.

Rule 1: The Probability Rule: The probability rule requires that I be able to characrerize any

alternative I face in a decision to my satisfaction by introducing uncertain distinctions of various

kinds and degrees and assigning probabilities to them. Once I have done this for all alternatives I

wish to consider within my present frame, I have completed the requirements of the probability

rule.

Rule 2: The Order Rule. The order rule requires that for each alternative I face, I construct the

possible futures formed by selecting the alternative and then one degree of each distinction used

to describe it. I call the result a "prospect". Sequential decisions following revelation of

information require prospects that describe each alternative and the distinctions that follow it.

When I have a complete list of prospects formed by the alternatives and their possible

consequences, I must then order them in a list, starting with the one I like best at the top and the

one I like worst at the bottom. I may have one or more prospects at the same level in the list. It is

possible that when I attempt to create this ordering, I find it difficult because I discover


uncertainties that I did not represent in the probability rule. This means that I must return to the

probability rule, add distinctions to represent them, and then repeat the process. Notice that the

order rule is an ordering of prospects in a certain world: no consideration of uncertainty is

allowed.

Rule 3: The Equivalence Rule. The equivalence rule applies, in principle, to any three prospects

at different levels in the list. Suppose I like prospect A better than prospect B, and I like prospect

B better than prospect C. The equivalence rule requires that I be able to assign a probability of

the best prospect A and one minus that probability of the worst prospect C such that I would be

indifferent to receiving this probability mixture of the best and worst prospects on the one hand

and the intermediate prospect B for certain on the other. We shall call the probability that

establishes this indifference a "preference" probability because it is not the probability of any

uncertainty that the clairvoyant could resolve.

When satisfying the equivalence rule, you may that find some of the prospects ordered in the

order rule require refinement to allow the proper assessment of preference probabilities. For

example, I may know that I prefer a steak dinner to a lamb chop dinner to a hamburger dinner.

However my preference probability for a steak dinner versus a hamburger dinner that would

make me just indifferent to a lamb chop dinner may well depend on further specification of each

dinner, a task that will require returning to the probability rule and creating new distinctions.

Since an actual decision could have hundreds if not thousands of prospects, and three at

different levels could in such cases be chosen in even more ways then there are prospects, you

can see why we said that the equivalence rule must apply in principle. Rarely will we have to

carry out the assignment of so many preference probabilities.

Rule 4: The Substitution Rule. The substitution rule requires that


if I should face in life any of the situations for which I assessed a preference probability in

the equivalence rule, and

if I assign a probability to receiving the best of the three prospects rather then the worst that

is equal to the preference probability I assigned, then

I remain indifferent between receiving the uncertain deal and the intermediate prospect.

The import of this rule is that the preference among uncertain prospects expressed in the

equivalence rule is not some hypothetical preference, but one that reflects my actual preferences

for uncertain deals. This means that probabilities and preference probabilities may be used

interchangeably in the analysis of decision problems.

Rule 5: The Choice Rule: The choice rule applies whenever I have two prospects at different

levels in my ordered list. If I prefer prospect A to prospect B, and if I face two alternatives with

different probabilities of only those two prospects, then I must choose the alternative with a

higher probability of receiving prospect A. In other words, I must choose the alternative with a

higher probability of the prospect I like better.

Note that this is the only rule that specifies the action I must take, and that it is so self-evident

that if I told someone I was violating it, they would think they had misheard me. Every aspect of

this choice is under my control: my preference and my probability.

I can make any decision using only these five rules. You can think of the process as

transforming an opaque decision situation into a transparent one by a series of transparent steps.

The transparent steps are the rules applied systematically. Using the first four rules, the choice

between any two alternatives can be reduced to an application of the choice rule. If there are

several alternatives, the repeated application of the choice rule will order them. (Howard, 1998).

The application of the rules simplifies considerably if I can describe all the prospects

completely in terms of a value measure, money. If I prefer more money to less, as I do, the rules


require that I be able to construct a nondecreasing curve on the money axis, which I shall call the

u-curve and any point on it a u-value. This curve summarizes my preferences for receiving

different amounts of the value measure with different probabilities, my risk preference.

The rules require that I order the alternatives in terms of their mean u-values, the sum of the

u-values of each prospect multiplied by the probability of that prospect. This computation yields

the u-value of any alternative. The certain equivalent of the alternative is then the amount of

money whose u-value equals the u-value of the alternative. Rather than having to be concerned

with assigning the preference probabilities in the order rule to the many possible threesomes of

prospects at different levels, I need only deal with the u-curve. We shall have more to say on the

terminology for this curve later.

If there is no uncertainty in a decision situation, the rules are considerably simplified. The

probability rule does not require any assignment of probabilities since every alternative now has

a certain consequence. The order rule is still required because it expresses preferences in a

deterministic world. There is no need for either the equivalence or substitution rules because they

or concerned with preferences when there is uncertainty. The choice rule would still come into

play; however, the probabilities involved would be 1 and 0. For example, would I rather have

$20 or $10?

The rules are required for systematic decision-making in an uncertain world. Simpler rules,

some of which we shall later discuss, cannot handle uncertainty.

Practice of Decision Analysis


The purpose of decision analysis is to achieve clarity of action. If you already know what to do

beyond any doubt, do it. If you do not know what to do, then apply the philosophy of decision

analysis at an appropriate level. The process can be as simple as realizing that you are incorrectly

including sunk costs or failing to recognize an alternative available to you.

Decision Engineering

The professional practice of decision analysis is decision engineering. The rules dictate the

norms, but not how to create the representation of a decision that will skillfully and efficiently

yield clarity of insight to the decision maker. What often happens when people try to analyze

decisions using a structure like a decision tree is that they are tempted to include every possible

uncertainty they can think of and thereby create an unanalyzable bush rather than the spare

structure desired.

Creating a focused analysis requires the continual elimination of every factor that will not

contribute to making the decision. This winnowing has been a feature of decision analysis since

the beginning (Howard, 1968, 1970). Since DAADT, the process has been described as a

decision analysis cycle, depicted in Figure 3.4 (Howard, 1984a).

Figure 3.4 Early Decision Analysis Cycle


A brief description of the cycle is this. After the framing the problem and specifying

alternatives, the uncertainties that appear to have an effect upon the decision are given nominal

ranges. The deterministic phase explores the sensitivity of alternatives to these uncertainties to

determine which are worthy of probabilistic analysis. The probabilistic phase encodes probability

distributions on these uncertainties, including necessary conditional distributions. It also requires

assessing the risk preference to be used for the decision. At this point, the best alternative can be

determined, but the process continues to the informational phase to find the value of eliminating

or reducing any or all of the uncertainties. The result is not only the best decision up to this point,

but also clear knowledge of the cost of ignorance. This may lead to new information gathering

alternatives and a repeat of the cycle, or simply to action.

By the way, this process was used in DAADT. Probability assignments were needed for a

material lifetime and three experts were knowledgeable. The probability distributions they

assigned individually and the one they agreed on collectively are shown in Figure 3.5.


Figure 3.5 Probability Assignment

It turned out that the effect of this uncertainty, deemed of great concern at the start of the

analysis, could be minimized by design changes and that the decision hinged on time preference.

There have been many refinements to the cycle over the years (Howard, 1984a, 1988). For

example in preparing for probabilistic evaluation, the “tornado diagram” (so named by a client

observing its shape) shows the sensitivity to uncertainties by the width of horizontal bars,

ordered from the widest to the narrowest. Since the variance of the resulting payoff usually

grows as the square of the length of the bars, only a few uncertainties with the longest bars are

typically needed in the probabilistic evaluation.


Influence, Relevance, and Decision Diagrams

A development that has aided practice and research over the past three decades is the

introduction of diagrams that contribute to the understanding, communication, and computation

of decision problems. The influence diagram (Howard and Matheson, 1980) provides a structure

that on the one hand is readily understood by decision-makers and yet is formally defined so that

it can serve as the basis for machine computation. I have found it to be an invaluable tool in the

classroom and in executive conversations.

Unfortunately, the use of the word "influence" has led some people into difficulty in creating

the diagrams. I have found it useful to introduce other forms. I call the special form of the

influence diagram that contains only uncertainties represented by uncertain nodes (usually

represented by a circle or oval) a "relevance diagram" (Howard, 1989, 1990). Relevance

diagrams directly address the question of inference in probabilistic networks, by representing the

conditional probability structure of distinctions. Arrows between uncertainties represent the

possibility that the probability of the successor uncertainty is conditional on the originating

uncertainty. Every relevance diagram expresses an assessment order for all uncertainties in the

diagram. A missing arrow is an assertion of the irrelevance of one uncertainty to another given

all uncertainties that are predecessors to both. Many puzzling problems in probability become

transparent when viewed in the form of a relevance diagram.

A decision diagram contains decision nodes (rectangles) and the value node (hexagon or

octagon). It may also contain one or more relevance diagrams. Arrows into a decision node are

called "informational" arrows; they signify that the node originating the arrow is known when

that decision is made. Arrows into the value node are called "functional" arrows; they show the


nodes on which value depends. Sometimes a special case of the uncertain node called a

deterministic node (double walled circle or oval) is created. The arrows into such a deterministic

node are functional arrows; the value of the node is computable from its inputs. You can

consider the value node as a special deterministic node.

Figure 3.6 shows a decision diagram for a test that is relevant to an uncertainty. The test costs

money; value resides in the decisions and the uncertainty.

ValueUncertaintyReport DecisionBuy Test?

Test Result

ValueUncertaintyReport DecisionBuy Test?

Test Result

Figure 3.6 A Decision Diagram

Note that this decision diagram contains a relevance diagram on Test Result and Uncertainty

that has no arrows entering it . Decision diagrams that contain relevance diagrams with this

property are said to be in canonical form.

Why is canonical form important? Arrows that go from a decision node to an uncertain node

are called "influence" arrows. They assert that the probability assignment to an uncertain node

depends upon how a decision is made. They are problematical because they blur a separation we

have made between actions that are under our control, alternatives, and the uncertainties that

might be resolved by a clairvoyant that are not under our control. While influences do not pose

significant problems in computing the best decision, they do increase the difficulty of computing

the value of clairvoyance (Howard, 1990). The simple reason is that if a clairvoyant could tell

you something about an uncertainty affected by an influence, he would be telling you something

about an action on which you have not yet decided and thereby would call into question your


free will. One can avoid this difficulty by eliminating all influences from the decision diagram,

thus placing it in canonical form. If necessary, this can be done by creating more nodes in the

relevance diagram conditioned on the influencing decisions. If this has been done, or if there are

no influences in the original diagram, then we say that the diagram is in canonical form.

Options

I consider one of the most important concepts in the practice of decision analysis to be that of an

option, properly defined (Howard, 1996). I do not restrict the term to financial options or even to

so-called real options that people attempt to replicate by portfolios of marketed securities. By

option, I mean an alternative that provides a new decision situation after the revelation of

information. Thus obtaining clairvoyance is an option because it is permitting you to make a

decision you face after resolving one or more uncertainties. The option may be obtained only at a

price, and there may be an additional cost, the exercise price, of using any alternative after

receiving the information. In computations involving clairvoyance, we typically assume that the

same alternatives will be available after the information is provided -- although this is not

necessary, we might have more or fewer. We also assume that there will be no additional cost to

exercising any of our original alternatives. All these comments apply to the option of performing

an information gathering activity, like an experiment, since we know that the experiment can be

valued by valuing clairvoyance on its results.

Sometimes options must be bought, like a fire extinguisher for your car; sometimes, they are

free of additional cost, like choosing what movie to see after driving to a multi-screen cinema.

Failure to recognize options and to incorporate them as sequential decisions is one of the

most important and consequential mistakes of decision analysis.


Cogency versus Verisimilitude

In representing a decision of professional size there is sometimes controversy about how much

detail to include. Notice that the standard of excellence for model representations, like model

trains, is verisimilitude: the correspondence of the model in detail to the real situation it is

intended to represent. At a model train exposition you might notice a tiny passenger train with a

bar car. In examining the bar car you might see that there are people inside holding drinks; this is

surely a detailed model. Further scrutiny shows that one of the drinks is clearly a martini

containing an olive; this will be a prize-winning train. Using a magnifying glass, you see that the

olive contains a pimento. Surely, this train will be "best of show".

Is verisimilitude the criterion for decision models? Is a decision model that includes "the

sales tax in Delaware" better than one that does not? The answer is no, unless that factor is

material to the decision. The criterion for decision models is cogency: whether the model leads to

crisp clarity of action for the decision maker. You should eliminate any feature that does not

contribute to this goal. If the decision maker insists on adding such embellishments, they should

be regarded as a professional courtesy, like giving the decision maker a ride to the airport, rather

than as part of professional decision analysis.

Ethics

Decision analysis is amoral, like an adding machine. Like any other powerful tool, people can

use decision analysis for good or ill. They can justify any course of action by manipulating the

elements of the analysis: the alternatives, information, and preferences. As organizations

increasingly accepted decision analysis, I became concerned about its ethical use (Howard, 1980,

1991, 2001), as should anyone teaching or practicing the discipline. The study of decision

analysis is an excellent precursor to ethical discussions for it illuminates both utilitarian


(consequence-based) and formalist (action-based) ethics. We find that characterizing actions as

to whether they are prudential, legal, and ethical is a helpful step in resolving ethical choices.

Language of Decision Analysis

Once again, decision analysis is more about clear thinking than about any of its detailed

procedures. Since even when thinking about a decision by ourselves we are going to use a

language to help us, it is extremely important that the language contain the proper concepts. This

is even more essential if we are discussing our decision with others. Whether supported by

modeling and computation or not, the decision conversation will become the basis for action.

My concern with the language of decision analysis goes back to DAADT:

One aid in reducing the problem to its fundamental components is restricting the

vocabulary that can be used in discussing the problem. Thus we carry on the discussion in

terms of events, random variables, probabilities, density functions, expectations,

outcomes, and alternatives. We do not allow fuzzy thinking about the nature of these

terms. Thus “The density function of the probability” and “The confidence in the

probability estimate” must be nipped in the bud. We speak of “assigning,” not

“estimating,” the probabilities of events and think of this assignment as based on our

“state of information.” These conventions eliminate statements like the one recently made

on a TV panel of doctors who were discussing the right of a patient to participate in

decision making on his treatment. One doctor asserted that the patient should be told of

“some kind of a chance of a likelihood of a bad result.” I am sure that the doctor was a

victim of the pressures of the program and would agree with us that telling the patient the


probability the doctor would assign to a bad result would be preferable.

Some of the communication advantages in using decision analysis were also spelled out:

One of the most important advantages of decision analysis lies in the way it encourages

meaningful communication among the members of the enterprise because it provides a common

language in which to discuss decision problems. Thus engineers and marketing planners with

quite different jargons can appreciated one another’s contributions to a decision. Both can use

the decision-analysis language to convey their feelings to management quickly and effectively.

My concern for precise language continues to the present. Refer to a recent paper on this

subject (Howard, 2004) for a thorough discussion of this issue. My intention is to have the

language we use in describing decisions consist of the simplest, least confusing, most accurate

terms for the concepts under discussion. Table 3.1 summarizes some of the recommended

changes in language that I have found useful in dealing with students and clients.

Table 3.1 Suggested Terminology Changes

Conventional Term Preferred Term Purpose of Change

dependence relevance To emphasize the informational rather than the causal nature of conditional probability assignments

outcome prospect To emphasize that decisions choose uncertain futures rather than only an immediate result.

expected value mean To recognize that the expected value is seldom to be expected. In discussing probability distributions, use “mean” to describe

the first moment.

expected value certain equivalent

To recognize that the expected value is seldom to be expected. In describing the value of an alternative with uncertain values,

use certain equivalent

utility u-curve To avoid confusion with other uses of the word utility in

related subjects, like marginal utility. The u-curve says nothing in a deterministic world.


Confusions

Uncertainty about Probability: As we previously discussed, the language of probability is

sufficient for describing the phenomenon of uncertainty. Yet people keep trying to build another

level they might call uncertainty about probability to describe any uneasiness they might feel in

the process of probability assignment. This is akin to the previous quote from DAADT where the

doctor said that the patient should be told of “some kind of a chance of a likelihood of a bad

result.” (Howard, 1988a). Once you have internalized the thinking of Laplace and Jaynes, any

notion of uncertainty about probability becomes unnecessary.

Deal Characterization and Deal Desirability: I recently consulted with a Board of Directors

concerned about whether to follow an alternative that had highly uncertain value prospects that

ranged from great profits to serious losses. There had been much previous discussion about

whether this alternative was "too risky" for the company. The members of the board were highly

educated and experienced business people.

The board might see an alternative like this as too risky for two logically separate reasons.

There might be a belief that the chances of large profits were overstated, or perhaps the chances

of losses understated. This would be a concern about the characterization of the deal, about

whether the analysis assigned proper probabilities to each level of profit and loss.

Once the deal has a proper characterization, the question is whether it is attractive given the

company's attitude toward risk: does the company have the stomach for this deal? Failure to

separate issues of characterization and desirability has been a perpetual source of confusion.

From DAADT:

“Often arguments over which is the best decision arise because the participants do not realize

that they are arguing on different grounds. Thus it is possible for A to think that a certain

alternative is riskier than it is in B’s opinion, either because A assigns different probabilities to


the outcomes than B but both are equally risk-averting, or because A and B assign the same

probabilities to the outcomes but differ in their risk aversion. If we are to make progress in

resolving the argument, we must identify the nature of the difficulty and bring it into the open.

Similar clarifications may be made in the areas of time preference or in the measurement of the

value of outcomes.”

Challenges of Decision Analysis

Classical Statistics Persists

In spite of the clear perspectives on uncertainty provided by Laplacian probability, much of the

teaching about uncertainty takes place in statistics classes where students are taught concepts that

can only confuse them. They learn about confidence intervals and confidence statements and

come to believe that a confidence statement is a probability assignment, even though the

professor is careful not to say so. If ever there was a "wolf in sheep's clothing" it is a confidence

statement posing as a probability assignment. Other classical statistics concepts have similar

problems. Hypothesis testing tells you nothing about how to form the hypothesis or how to

determine the probability level at which it can be rejected. Furthermore, if a hypothesis is

rejected, there is no procedure for what to do next. Maximum likelihood procedures are based

only on the data from an experiment and have no place for any preceding knowledge. None of

these methods are what Maxwell had in mind."

Dealing with Multiple Attributes


Decision problems can have prospects with many attributes that the decision maker would like to

consider. We shall here consider some of the procedures that have been developed to address this

problem.

Direct and Indirect Values: Sometimes the decision maker needs to assess preferences on many

fewer attributes than he or she would think. To see why, let us define direct and indirect values.

A direct value is one to be traded off by the decision maker against other direct values. An

indirect value is a distinction that is relevant to a direct value, but is not a direct value itself.

To illustrate, let us consider preferences for tourism and wildlife in Africa. A resort hotel

operator might have values like those shown on the left of Figure 3.7.


Value

TourismWildlife

IndirectDirect

Value

TourismWildlife

IndirectDirect

Environmentalist’s Values Environmentalist’s Values

Value

TourismWildlife

DirectDirect

Value

TourismWildlife

DirectDirect

Two Direct Values

Value

Tourism

Figure 3.7 Values for Tourism and Wildlife

We suppose that he does not care at all about wildlife, but is very concerned about tourism.

He places a direct value on tourism and an indirect value on wildlife. Notice that the arrow

connecting wildlife and tourism in this diagram is a relevance arrow. This means that the

importance of wildlife depends on the information of the hotel operator. If a convincing study

showed that tourism would be unaffected by the amount of wildlife, then the operator would not

have an indirect value on wildlife. But if he believes, as is likely to be the case, that the tourists

come to see wildlife, then he would support measures preserving wildlife habitat and the

prevention of poaching.

The center diagram in Figure 3.7 shows how an environmentalist might value the same

distinctions. The environmentalist places a direct value on wildlife and an indirect value on

tourism. The importance of tourism to the environmentalist will depend on his beliefs about how

the presence of tourists enhances or harms wildlife.

Notice that both of these people have only one direct value and therefore no reason for value

trade-offs. Alternatives are judged by their effect on the one direct attribute. The right diagram in

Wildlife

Value

Tourism Wildlife

Direct Indirect Direct Indirect

Hotel Operator’s Values


Figure 3.7 shows a value function for someone who places a direct value on both wildlife and

tourism. Alternatives that produced different levels of these attributes would require trade-offs

between them.

I once conducted a session with oil company executives who believed they had to deal with

about 30 different attributes in making their decisions. After about an hour of discussion, and of

direct and indirect values, they finally agreed that there were only two direct values. One was the

profitability of the company, and the other was harm to people surrounding their facilities as the

result of company operations.

Focusing on direct values considerably simplifies analysis of multiattribute decision

situations.

Treating Multi-Attribute Problems Using a Value Function. Suppose I have reduced a multi-

attribute problem I face to n attributes that have direct value. I would want one of these attributes

to be a value measure so that I can compute the value of clairvoyance, or of any experimentation,

in terms of this measure. In this n-dimensional space I now construct iso-preference surfaces,

combinations of attribute levels that are equally desirable to me. I can then identify each surface

by its intercept with the value function. If I now have a joint distribution on the n-1 other

attributes, I will have a derived distribution on the value measure. I can then assign a u-curve on

the value measure and determine the certain equivalent of any alternative that could produce

these attributes, and thereby make my choice. The u-curve on the value measure would imply

preferences under uncertainty for each of the other attributes. No additional information would

be contained in these preferences. Proceeding in this fashion will allow establishing trade-offs

within the attributes and in particular, between any two attributes when the value of all others are

specified. The incorporation of the value measure permits the decision maker to see the

implication of choices in terms of a value scale of common experience. The benefit of being able


to compute the value of clairvoyance or of any other information is attainable only by using a

value measure (Matheson and Howard, 1968).

Other Approaches to Multi-Attribute Problems. Other approaches for multi-attribute problems

divide into two classes. Those that satisfy the rules and those that do not. One that satisfies the

rules is the approach of placing a ‘multidimensional utility function’ directly on the attributes

(Keeney and Raiffa, 1976). This approach does not use a value function and as a result cannot

have the advantage of computing the value of clairvoyance unless a value measure is one of the

attributes. A check of the reference (Keeney and Raiffa, 1976) reveals that there is no discussion

of the value of information gathering in the book.

The other class of approaches to the multi-attribute valuation problem consists of

simplifications that do not offer all the benefits of the rules or methods that do not follow the

rules and hence may not meet some of the desiderata.

Weight and Rate. A simple way to handle many attributes is to assign, say, 100 points total and

then assign them to each of the attributes in accordance with their importance. The next step is to

rate each of the alternatives by seeing how many of the points of each attribute or earned by that

alternative. The point value of the alternative is obtained by summing over all attributes. This

procedure is very simple and may be helpful in choosing a car or a stereo. It is not so helpful

when there is uncertainty. Weight and rate assumes that preference for these deterministic

attributes is linear in the attributes: there can be no interaction among them. When I construct my

preferences for a peanut butter and jelly sandwich, I find they do not meet this condition. In

summary, weight and rate methods can be helpful, but cannot bear the weight of decisions with

uncertainty like choosing treatment by drugs or an operation, pharmaceutical development, or

planning finances for retirement.


Analytic Hierarchy Process. A widely used process that does not obey the rules is the analytic

hierarchy process (Howard, 1992). The result of the process is a weight and rate system that is

derived on the basis of comparative judgments of importance, preference, and likelihood. The

process has an air of mathematical sophistication, but its results are easily produced in a

spreadsheet by averaging or, equivalently, and with much more ado, by eigenvalues from matrix

iteration. AHP can incorporate uncertainty only approximately, has a major difficulty in

incorporating experimental information, cannot compute the value of information, and can

provide no warranty that the alternative it recommends is the best one. Since it does not follow

the rules, it is subject to failing a desideratum, like the one requiring that removing a

noninformative alternative cannot change the ordering of the existing alternatives (Dyer, 1990).

Why, then, do inferior processes find favor with decision-makers? The answer is that they do

not force you to think very hard or to think in new ways. Since we rarely find epistemic

probability in our educational system, even in engineering, medical, and business schools, it is

not surprising that people generally find it challenging to follow the dictum of Maxwell. In

decision-making, as in many other pursuits, you have a choice of doing something the easy way

or the right way, and you will reap the consequences.

Risk Preference

The notion of risk preference and its representation is still a problem for people and

organizations. I once heard distinguished decision science professor give a brilliant presentation

to business school professors on the necessity of having a personal risk attitude to guide

decisions. After the lecture, and in private, a graduate student asked the professor what his

personal risk attitude, i.e., u-curve, was. The professor admitted that he did not have one.

The issue of risk preference in my experience goes back to DAADT:


… for example, although we have tended to think of the utility theory as an academic

pursuit, one of our major companies was recently faced with the question, “Is 10 million

dollars of profit sufficient to incur one chance in I million of losing I billion dollars?~’

Although the loss is staggering, it is realistic for the company concerned. Should such a

large company be risk-indifferent and make decisions on an expected value basis? Are

stockholders responsible for diversifying their risk externally to the company or should

the company be risk-averting on their behalf? For the first time the company faced these

questions in a formal way rather than deciding the particular question on its own merits

and this we must regard as a step forward.

Life and Death Decisions

One area in which I have a special interest is the use of decision analysis for making safety

decisions, and in general decisions involving a risk of death or serious bodily harm (Howard,

1978, 1980a, 1984, 1989a, 1999). I distinguish three stages of analysis. The first is risk

assessment to assess the magnitude of risks to life in a proper unit. The second is risk evaluation

to determine the importance of the risk in monetary terms. The third is risk management, or

decision-making, to choose what course of action best balances the advantages, disadvantages,

and safety consequences.

By using a properly sized probability unit, the microprobability, defined as a probability of

one in one million, small risks of death can be appreciated. A micromort, one microprobability of

death, is a handy unit of measure for the death risks faced from accidents. Placing a value on a

micromort permits making many safety decisions such as whether to take a beneficial drug with

possible deadly side effects.


Future of Decision Analysis

Is it possible that the discipline of decision analysis developed over the last 40 years is no longer

necessary because of improvements in decision-making? Has the phenomenal growth in

computation and the availability of information obviated the need for decision analysis?

Unfortunately, that is not the case. Executives today are making the same mistakes their parents

and grandparents used to make. I hear consultants who observe poor executive decision making

say, "If we could have only 1% of the waste." Even the field of science has not learned the

lesson. Scientific journals are still accepting hypotheses that cannot be rejected at the 95%

confidence level. I believe that if Laplace could see the state of modern decision-making, he

would be appalled by our failure to use the systems we have been discussing, especially now that

we have the computational and communication tools he could only have dreamed of. With few

exceptions (Decision Education Foundation), students in elementary, secondary, college and

graduate schools do not learn how to think in an uncertain world.

Decision analysis has thrived in certain environments. Some consulting companies rely

extensively upon it. Decision analysis is entrenched in major industries like petroleum and

pharmaceuticals, and is heavily employed in electric power. There is hardly an industry from

paper to moviemaking that has not made use of decision analysis.

It has been said that every strength is accompanied by a weakness, and that is true of decision

analysis. One of its greatest strengths is its transparency: the decision basis is laid out for all

participants to see -- the alternatives considered, the information used and its sources, and finally

the preferences. Organizations wanting to use the full capability of their members to improve the

quality of a decision find this to be a great advantage. However, transparency is equally a threat


to organizations that wish to limit alternatives, control information, and hide preferences. The

more open the organization, private or public, the more it will value the process of decision

analysis.

Epilog

To me, incorporating the principles and philosophy of decision analysis is not just learning the

subject, but more like installing a new operating system in your brain.

Acknowledgment

I thank Ali Abbas for many helpful suggestions in preparing this chapter.


References

Bernoulli, D. (1738). “Specimen Theoriae Novae de Mensura Sortis”( Exposition of a New

Theory on the Measurement of Risk) Commentarii Academiae Scientiarum Imperialis

Petropolitanae, Tomus V [Papers of the Imperial Academy of Sciences in Petersburg, Vol. V]

,pp. 175–192.

Cox, R.T. (2001). Algebra of Probable Inference. The Johns Hopkins University Press

Decision Education Foundation. http://decisioneducation.org

Dyer, J.S. (1990). Remarks on the Analytic Hierarchy Process. Management Science, Vol. 36,

No. 3, (249-258).

Howard, R.A. & Matheson, J.E. (1980)."Influence Diagrams", Private Report. Reprinted in

Howard, R.A. & Matheson, J.E. (Eds) READINGS on the Principles and Applications of

Decision Analysis. Strategic Decisions Group, Menlo Park, California, April 1984. To appear

in the journal Decision Analysis.

Howard, R.A. (1965). Bayesian Decision Models for Systems Engineering. IEEE Transactions

on Systems, Science and Cybernetics, Vol. SSC-1, No. 1, November

Howard, R.A. (1966). Decision Analysis: Applied Decision Theory. Proceedings of the Fourth

International Conference on Operational Research, pp. 55-71 , Wiley-Interscience,.

Reprinted in Howard, R. A. & Matheson, J. E. (Eds.) READINGS on the Principles and

Applications of Decision Analysis. Strategic Decisions Group, Menlo Park, California.

Howard, R.A. (1966a). Information Value Theory. IEEE Transactions on Systems, Science and

Cybernetics, Vol. SSC-2, No. 1, August. (22-26)

Howard, R.A. (1968). The Foundations of Decision Analysis. IEEE Transactions on Systems,

Science and Cybernetics, Vol. SSC-4, No. 3, pp, September. (211-219).

http://www.amazon.com/exec/obidos/search-handle-url/index=books&field-author-exact=Richard%20T.%20Cox/002-1320802-8227200


Howard, R.A. (1970). Decision Analysis: Perspectives on Inference, Decision and

Experimentation. Proceedings of the IEEE, Vol. 58, No. 5, May 1970 (632-643).

Howard, R.A. (1973). Decision Analysis in Systems Engineering. In Miles, Jr., R. F. (Ed.)

Systems Concepts: Lectures on Contemporary Approaches to Systems., Wiley-Interscience.

pp. 51-85.

Howard, R.A. (1975). Social Decision Analysis. Proceedings of the IEEE, Vol. 63, No. 3, 359-

371.

Howard, R.A. (1978). Life and Death Decision Analysis. Proceedings Second Lawrence

Symposium on Systems and Decision Sciences, Berkeley, California, October.

Howard, R.A. (1980). An Assessment of Decision Analysis. Special Issue on Decision Analysis,

Operations Research, Vol. 28, No. 1, January-February 1980, (4-27).

Howard, R.A. (1980a). On Making Life and Death Decisions. In Schwing, R. C. & Albers, Jr.,

W. A. (Eds.) Societal Risk Assessment, How Safe Is Safe Enough?, General Motors Research

Laboratories, Plenum Press,.

Howard, R.A. (1984). On Fates Comparable to Death. Special Issue of Management Science,

April.

Howard, R.A. (1984a). The Evolution of Decision Analysis. In Howard, R. A. & Matheson, J. E.

(Eds.) READINGS on the Principles and Applications of Decision Analysis (pp. 7-16),

Strategic Decisions Group, Menlo Park, California.

Howard, R.A. (1988). Decision Analysis: Practice and Promise. Management Science, Vol. 34,

No. 6, pp., June 1988. (679-695)

Howard, R.A. (1988a). Uncertainty About Probability: A Decision Analysis Perspective. Risk

Analysis, Vol. 8, No. 1, (91-98).

Howard, R.A. (1989). Knowledge Maps. Management Science, Vol. 35, No. 8, August. ( 903-


922.)

Howard, R.A. (1989a). Microrisks for Medical Decision Analysis. International Journal of

Technology Assessment in Health Care, Vol. 5, No. 3, (357-370)

Howard, R.A. (1990). From Influence to Relevance to Knowledge. In Oliver, R. M. & Smith,

J.Q. (Eds.) Influence Diagrams, Belief Nets and Decision Analysis (pp. 3-23). John Wiley &

Sons Ltd.

Howard, R.A. (1991). Business Ethics: Tell the Truth. Journal of Management Development,

Vol. 11, No. 4, (4-10).

Howard, R.A. (1992). Heathens, Heretics and Cults: The Religious Spectrum of Decision

Aiding. Interfaces, Vol. 22, No. 6. (15-27).

Howard, R.A. (1992a). In Praise of the Old Time Religion. In Edwards, W. (Ed.). Utility

Theories: Measurements and Applications. Kluwer Academic Publishers. , pp. 27-55

Howard, R.A. (1996). Options. In Zeckhauser, R., Keeney, R., and Sibenius, J. (Eds.) Wise

Choices: Decisions, Games, and Negotiations. Boston: Harvard Business School Press.

Howard, R.A. (1998). Foundations of Professional Decision Analysis: A Manuscript in Process.

Stanford Course Notes.

Howard, R.A. (1999). Life and Death Decisions for Individuals and Couples. In Shanteau, J.,

Mellers, B., and Schum, D (Eds.) Decision Science and Technology: Reflections on the

Contributions of Ward Edwards. Boston: Kluwer Academic Publishers.

Howard, R.A. (2000). Decisions in the Face of Uncertainty. In Alexander, C. (Ed.) Visions of

Risk. London: Pearson Education Limited

Howard, R.A. (2001). The Ethical OR/MS Professional. INTERFACES, Vol. 31, No. 6,

November-December, (69-82)

Howard, R.A. (2004). Speaking of Decisions: Precise Decision Language. Decision Analysis,


Vol. 1, No. 2, June 2004.).

Jaynes, E.T. (1986). Bayesian Methods: General Background. In Justice, J. H. (Ed.) Maximum-

Entropy and Bayesian Methods in Applied Statistics. Cambridge Univ. Press, Cambridge, p.

1.

Jaynes, E.T. (2003). Probability Theory : The Logic of Science. Cambridge University Press

Jeffreys, H. (1939). The Theory of Probability. Oxford University Press, New York (3rd Ed.,

1961)

Keeney, R.L. & Raiffa,H. (1976). Decisions with Multiple Objectives. John Wiley & Sons, Inc...

Laplace, P.S. (1812). Analytical Theory of Probability

Laplace, P.S. (1996). A Philosophical Essay on Probabilities. Dover Publications.

Matheson, J.E., &. Howard, R.A (1968). An Introduction to Decision Analysis In The Principles

and Applications of Decision Analysis, Vol. I,. Howard, R. A., and Matheson, J. E. (Eds.).

Strategic Decisions Group, Menlo Park, California, USA.

http://www.amazon.com/exec/obidos/search-handle-url/index=books&field-author-exact=E.%20T.%20Jaynes/002-1320802-8227200

http://www.amazon.com/exec/obidos/search-handle-url/index=books&field-author-exact=Pierre%20Simon%20Laplace/002-1320802-8227200

the foundations of decision analysis...

Documents