i. glymour's bootstrapping theory of …mcps.umn.edu/philosophy/10_1glymour.pdf · ·...

I. GLYMOUR'S BOOTSTRAPPING THEORY OF CONFIRMATION

This page intentionally left blank

Clark Glymour-

On Testing and Evidence

I . Introduction

If we knew the probabilities of all things, good thinking would be aptcalculation, and the proper subject of theories of good reasoning would beconfined to the artful and efficient assembly of judgments in accordancewith the calculus of probabilities. In fact, we know the probabilities of veryfew things, and to understand inductive competence we must constructtheories that will explain how preferences among hypotheses may beestablished in the absence of such knowledge. More than that, we mustalso understand how the procedures for establishing preferences in theabsence of knowledge of probabilities combine with fragmentary knowl-edge of probability and with procedures for inferring probabilities. And,still more, we need to understand how knowledge of probabilities isobtained from other sorts of knowledge, and how preferences amonghypotheses, knowledge of probabilities, and knowledge of other kindsconstrain or determine rational belief and action.

For at least two reasons, the first and second of these questions havebeen neglected for many years by almost everyone except groups ofphilosophers who have been virtually without influence on practitioners of"inductive logic." One reason, unfortunately, is that a long tradition ofattempts to construct an account of hypothesis testing, confirmation, orcomparison outside of the calculus of probabilities has failed to achieve atheory possessing both sufficient logical clarity and structure to meritserious interest, and sufficient realism to be applied in scientific andengineering contexts with plausible results. A second reason is the recentpredominance of the personal or subjective interpretation of probability:probability is degree of rational belief, and rational inference consists inappropriate changes in the distribution of intensity of belief. Rationality atany moment requires only that belief intensities be so distributed as tosatisfy the axioms of probability. Thus, even when we reason about thingsin the absence of knowledge of objective chances, we still retain degrees of

3

4 Clark Glymour

belief, and hence if we are rational our reasoning is an application of thecalculus of probabilities or of principles for changing probabilities. Thecommunity of philosophical methoclologists has been captured by thisvision, and in consequence many have come to think that there is properlyno separate question regarding how hypotheses may be assessed in theabsence of knowledge of probabilities. I continue to disagree.

Theory and Evidence aimed to show that (and in some measure, why) thetradition of work on questions of testing and comparing hypotheses outsideof the calculus of probability had failed to produce a plausible and nontrivialtheory of the matter; quite as much, I aimed to argue that the Bayesianassimilation of scientific reasoning carried on without knowledge ofprobabilities to a species of probable reasoning is both inaccurate andunrevealing about the procedures that, in the absence of knowledge ofprobabilities, make for inductive competence. To no very great surprise,arguments on these scores have failed to persuade either the advocates ofhypothetico-deductivism or Bayesian confirmation theorists. On the posi-tive side, Theory and Evidence described and illustrated an elementarystrategy for testing and discriminating among hypotheses. Questionsregarding how such a strategy may be combined with fragmentaryknowledge of probabilities were addressed only briefly and in passing.

What follows is unpolemical. The characterization of the testing relationfor systems of equations and inequations was left undeveloped in Theoryand Evidence and was in effect given through a class of examples, andrather special examples at that, in Chapter 7 of the book. Here I have triedto give a more general characterization of the strategy, and to describe indetail how particular features may be varied. I have also tried to character-ize some of the structural relations that the account of testing permits us todefine, and to indicate some questions about these relations that deservefurther exploration. Theory and Evidence contained an abundance ofinformal illustrations of applications of the testing strategy. In this essay aquite different illustration, suggested to me by William Wimsatt, is given:It is shown that for systems of Boolean equations, certain variants ofbootstrap testing are equivalent to a formalized testing strategy used in thelocation of faults in computer logic circuits. Since the context is a veryspecial one, this is an interesting but not a distinctive result. A hypothetico-deductive approach will yield the same equivalence, assuming a specialaxiomatization of the system of equations.

This essay begins to discuss how to combine certain of the features of

ON TESTING AND EVIDENCE 5

bootstrap testing with fragmentary knowledge of probabilities. In reflect-ing on these questions, I have been driven to reliance on work by ArthurDempster and its generalization and reconception by Glenn Shafer.Shafer's interesting work has not found an eager reception among metho-dologists, but in certain contexts it seems to me to be just the right thing. Atthe very least, I hope the reader will come to see from the discussion belowhow elements of Shafer's theory of belief functions arise in a natural waywhen one considers evidence in combination with fragmentary knowledgeof probabilities. I have applied Shafer's theory in a "bootstrap" setting tothe same engineering problem discussed earlier—fault detection in logiccircuits.

The attempt in Chapter 5 of Theory and Evidence to characterize aconfirmation or testing relation for formalized theories in a first-orderlanguage is palpably clumsy. With five complex clauses, the analysis reads,as one commentator noted, like the fine print in an auto-rental contract. Itspeaks for the intellectual good will of many commentators that they havenonetheless patiently investigated its consequences and examined itsmotivations, I am especially indebted to David Christensen, Aron Edidin,Kevin Kelly, Robert Rynasiewicz, and Bas van Fraassen. Edidin showedthat the attempt to allow computations to entail the hypothesis to be testedwas unsound, and could not be realized without permitting trivial confir-mation relations. Van Fraassen saw the essential ideas were contained inthe applications to systems of equations, reformulated the ideas moreelegantly, and gave some intriguing yet simple examples. Rynasiewiczshowed that the rental clauses had loopholes. Christensen showed that thebootstrapping idea has a fundamental difficulty in the context of thepredicate calculus: any purely universal hypothesis containing only a singlenon-logical predicate can be "tacked on" to any testable theory in whichthat predicate does not occur, and the "tacked on" hypothesis will bebootstrap tested with respect to the extended theory. Although Kelly hasshown that Christensen s difficulty does not arise when the account oftesting is developed in the context of relevance logic, in what follows I haveapplied the testing strategy only to systems of equations and inequations.

2. Three Schemas for Bootstrap Testing of Equations andInequations

Let A be any algebra with a natural partial or complete ordering. I shallassume it is understood what is meant by an assignment of values in A to

6 Clark Glymour

variables satisfying an equation or inequality H, and what it means for H tobe valid. For algebraic equations or inequalities, H and K, over A, we shallsay that H entails K if every set of values satisfying H satisfies K.Analogously if H is a set of equations or inequations. A subtheory of asystem of equations or inequalities T is any system that is satisfiedwhenever T is satisfied. Two systems are equivalent if each is a subtheory ofthe other. A variable is said to occur essentially in an equation or inequalityH if that variable occurs in every system equivalent to H. Var (H) willdenote the set of variables occurring essentially in H. The simplest schemefor bootstrap testing is then:

Schema I: Let H and T be as above and mutually consistent; let e* be a set ofvalues for the variables e,, consistent with H and T. For each x( in Var(H)let Tj be a subtheory of T such that

(i) Tj determines xf as a (possibly partial) function of the Cj, denotedT / \

i(ej);

(ii) the set of values for Var(H) given by xf* = Tf(e*) satisfies H;(iii) if &T is the collection of Tf for all of the xf in Var(H), &T does not

entail that H is equivalent to any (in) equation K such thatVar(K) C Var(H);

(iv) for all i, H does not entail that T, is equivalent to any (in) equationK such that Var(K) C Var(Tj).

If these conditions are met, the e* are said to provide a positive test of Hwith respect to T. These are roughly the conditions given, for a special case,in Chapter 7 of Theory and Evidence and illustrated there in applications inthe social sciences. The motivation for each condition in Schema I isreasonably straightforward. The requirement that a value be determinedfor each quantity occurring essentially in H reflects a common prejudiceagainst theories containing quantities that cannot be determined from theevidence; more deeply, it reflects the sense that when values for the basicquantities occurring in H have not been determined from the evidenceusing some theory, then that evidence and the relevant fragment of thattheory do not of themselves provide reason to believe that those basicquantities are related as H claims them to be.

The reason for the second requirement in Schema I is obvious. The thirdrequirement is imposed because if it were violated there would be somequantity x occurring essentially in H such that the evidence and the methodof computing quantities in Var(H) from the evidence would fail to test the


constraints H imposes on x. Thus a significant component of what H sayswould go untested. The fourth condition is motivated by the fact, noted byChristensen, that the first three conditions alone permit a value of y andvalues of c and d to test the hypothesis x — y with respect to the pair ofequations x = y and c = d: simply measure y and compute x using theconsequence x = y 4- (c-d) and measured values of c and d.

The third condition might even be strengthened plausibly as follows:

Schema 11: Let H and T be as above and mutually consistent, and let e* be aset of values for the variables Cj consistent with H and T. For each x( inVar(H) let Tf be a subtheory of T such that

(i) Tj determines Xj as a (possibly partial) function of the Cj, denotedT / \i(ej);

(ii) the set of values for Var(H) given by xs* = Tj(e*) satisfies H;(iii) for each xs in Var(H), there exists a set of values e^ for the Cj such

that for k ± i, Tk(eD = Tk(e*) and the set of values for Var(H)given by x^ = T^e*) does not satisfy H;

(iv) for all i, H does not entail that Tj is equivalent to any equation Ksuch that Var(K) C Var(Tj).

When the conditions of Schema II are met, the values e* are again said toprovide a positive test of H with respect to T.

Schemas I and II are not equivalent, and Schema II imposes strongerrequirements on positive tests. For example, let H be x j = x2-x3.Suppose that the xs are measured directly, so that xf = e{ in Schemas I andII. Let the evidence be that X! = 0, \2 — 0, and x3 = 1. Then H ispositively tested according to Schema I but not according to Schema II.

The difference between these two schemas lies entirely in the thirdclause, for which there is a natural further alternative:

Schema HI

(iii) If K is any sentence such that &T entails K and Var(K) C Var(H),then K is valid.

Schema III is neither stronger nor weaker than Schema II. For theexample just given, X! = X2 = 0, and x3 = 1 does count as a positive testaccording to III. On the other hand, if we have

H: x > 3T: x = e2

8 Clark Glymour

and evidence e = 2, where variables are real valued, then we have a positivetest according to I and II but not according to III since T entails x > 0. Henceall three schemas are inequivalent. Further, Schema III satisfies aconsequence condition: If H is positively tested with respect to T, then so isany consequence G of H, so long as Var(G) C Var(H).

3. Variant Schemes3.1 Set Values

Schemas I, II, and III are stated in such a way that the hypothesis tested,H, may state an equation or an inequality. The T, in these schemas,however, must be equations. In situations in which the Tj includeinequations the schemas must be modified so that the variables become, ineffect, set valued. Rynasiewicz and Van Fraassen have both remarked thatthey see no reason to require that unique values of the computed quantitiesbe determined from the evidence, and I quite agree—in fact, in thelamented Chapter 8 of Theory and Evidence, the testing scheme forequations is applied without the requirement of unique values.

A set valuation for a collection of variables is simply a mapping takingeach variable in the collection to a subset of the algebra A. For any algebraicfunction f, the value of f(x) when the value of x is (in) A is (f(a) I a is in A} ,written f(A). If x; = TJ(CJ) is an equation, the value of the x} determined bythat equation for values of 6j in A j is the value of T,(ej) when the value of Cjis Aj. If Xj > Tj(ej) is an inequation, the value of xf determined by thatinequation for values of Cj in Aj, is {a in A : b is in Tj(Aj) implies a > b}.(There is an alternative definition {a in A I there exists b in Tf(Aj) such that b> a}). For a collection of inequations determining x; , the set-value of x(

determined by the inequations jointly is the intersection of the set-valuesfor Xj that they determine individually.

An equation or inequation H(XJ) is satisfied by set values Af for itsessential variables x( if there exists for each i a value x,* in A( such that thecollection of Xj* satisfy H in the usual sense. The notions of equivalence andentailment are explained just as before. With these explications, SchemasI, II, and III make sense if the e* are set values or if the T assertinequalities. (Of course, in the schemas we must then understand a value ofx to be any proper subset of the set of all possible values of x.)

3.2 Extending "Quantities"

The first condition of each schema requires that for H to be positively


tested, a value for each quantity occurring essentially in H must bedetermined. That is not a condition we always impose when testing ourtheories. We do not nowadays impose it when testing, say, hypotheses ofstatistical mechanics, where the quantities that experiment determines arenot, e.g., values of the momenta of individual molecules, but instead onlyaverages of large numbers of such values. Yet at the turn of the century it

was in considerable part the inability of statistical mechanicians todetermine values for such basic quantities reliably that gave anti-atomistssuch as Ostwald reason to doubt kinetic theory. Ostwald's methodologicalsensibilities were not, in that particular, idiosyncratic, and even his atomistopponents, such as Perrin, realized something of what was required todisarm his criticisms. I view Sehemas I, II, and III as providing accounts ofwhen, in the absence of special knowledge about the quantities occurringin an hypothesis, a piece of evidence is determined by a theory to bepositively relevant to the hypothesis. But sometimes—as with statisticalmechanical hypotheses nowadays—we do have special knowledge aboutthe quantities that occur in the hypothesis to be tested. We do not needsome special evidence that, for example, the value of the quantitydetermined to be the average kinetic energy really is the average of themany distinct quantities, the kinetic energies of the individual molecules.Sehemas I, II, and III can be altered to apply to situations in which we dohave knowledge of this sort.

Let f i (x j ) be functions of the collection of variables x( occurring essentiallyin H, and suppose the following hold:

(a) All f] occur in H.(b) No one occurrence of any quantity x, in H is within the scope of

(i.e., is an argument of) two distinct occurrences of functions in theset of f j .

(c) There exist values V for the f\ such that every set of values for the xf

consistent with V satisfies H, and there exist values U for the f]such that every set of values for the xf consistent with U fails tosatisfy H.

Call a set of variables or functions satisfying (a), (b), and (c) sufficient forH. Values V of the functions f] are said to satisfy H if every set of values forthe Xj in Var(H) consistent with V satisfies H. Analogously, values U for f[contradict H if every set of values for the x, in Var(H) consistent with U failto satisfy H. Now Schema I can be reformulated as follows:

t

10 Clark Glymour

Schema IV: Let H be an equation or inequation, let T be a system ofequations or inequations consistent with H, and let Q be a set ofvariables or functions sufficient for H. Let e* be a set of values for thevariables 6j consistent with H and T. For each f] in Q let T] be asubtheory of T such that:

(i) T] determines f] as a (possibly partial) function of 6j , denotedTi(ej).

(ii) the set of values for elements of Q given by fi* = T](e*) satisfiesH.

(iii) the collection of T] for fj in Q does not entail that H is equivalent toany (in) equation E such that Var(E) C Var(H).

(iv) for all 1, H does not entail that T] is equivalent to any (in) equationK such that Var(K) C Var(T,).

Schema IV does not satisfy the equivalence condition; hence a hypothe-sis K must be regarded as positively tested according to Schema IV if andonly if K is equivalent to a hypothesis H satisfying the Schema.

An analogous version of Schema II, call it Schema V, can readily bestated. I shall forbear from giving the whole of the thing, and instead only aversion of the third clause.

Schema V:

(iii) For each f] in Q, there exists a set of values e +, for 6j such that fork =£1, Tk(e + ) = Tk(e*), and the set of values for Q given by f^ =Tm(e + ) contradicts H.

The reformulation of Schema III is obvious.

4. Questions of Separability

I view the bootstrap testing of a theory as a matter of testing one equationin a theory against others, with an eye in theory construction towardsestablishing a set of equations each of which is positively tested withrespect to the entire system. Clearly, in many systems of equations thereare epistemological entanglements among hypotheses, and that fact consti-tutes one of the grains of truth in holism. Are there natural ways to describeor measure such entanglements?

Let E be a set of variables. I shall assume that all possible evidenceconsists in all possible assignments of values to variables in E. Explicitreference to E is sometimes suppressed in what follows, but it should beunderstood that all notions are relative to E. We consider some definite


system of equations or inequations, T. All hypotheses considered areassumed to be among the consequences of T. Again, even where not madeexplicit, notions in this section are relative to T and E.

One natural way of characterizing the epistemic interdependence ofhypotheses is through the following notions:

1. H is E-dependent on K if for every (possible) positive test of H, K isa consequence of the set of Tj used in the test,

2. H is cyclic if there is a sequence of hypotheses HI, . . . Hn, n > 2,such that Hj is inequivalent to H; + i and Hj is E-dependent onHj + !, for all i, and H! = Hn = H.

3. H E-entails K if every set of values of E that provides a positive testof H also provides a positive test of K.

4. H and K are interdependent if each is E-dependent on the other; E-equivalent if each E entails the other.

Notice that the notion of two equations being E-equivalent is not thesame as that of each being E-dependent on the other. For example, amongthe consequences of the theory

a = eib = e2

a = b

with ei and e2 directly measurable quantities, the equations ej = b ande2 = b are E-dependent on one another, but not on a = b, whereas allthree of these equations are E-equivalent. Each of the claims e! = b ande2 = b is cyclic. E-entailment relations have no neat or natural relation toordinary entailment when the notion of positive test is explicated accordingto Schema I or Schema II. For Schema III, however, I make the followingconjecture:

If X is a collection of equations or inequations, A is an equation orinequation such that every variable occurring essentially in A occursessentially in some member of X, and X entails A, then X E-entails A.

Thus we can understand E-entailment for Schema III as an extension ofordinary entailment over a restricted language.

From the point of view of methodology, E-dependence relations seemmore important than E-entailment relations, even though they are for-mally less interesting.

12 Clark Glymour

5. Fault Detection in Logic CircuitsA standard problem in computer science is that of determining faulty

circuit elements in a complex logic circuit from data as to circuit inputs andoutputs. William Wimsatt has remarked that this is in effect a problem oflocalizing confirmation (or disconfirmation) within a complex theory, sobootstrap techniques should apply to it. Indeed, because such circuits canbe represented by a system of equations with variables taking values in atwo-element Boolean algebra, bootstrap testing generates a technique forthe determination of faults. Computer scientists have developed methodsfor the solution of the problem, and although their methods may not look asif they are bootstrap tests, the two procedures are equivalent.

In the simplest way, bootstrap testing (in the sense of Schemas I or III),applies to logic circuits as follows: Consider a circuit such as the oneillustrated below:

The input signals are the x f , the output signal is z. Cusps signify an "or" or" + " gate, and dees signify an "and" or "•" gate. Each line of the circuit isan immediate function either of inputs or of lines to which it is connected bya gate. Thus expressing each line, and also the output z, as a function ofthose lines that lie to the left of it and are connected to it by a gate, or as afunction of the input variables where no gate intervenes, results in a system


of equations (e .g . , e = X ! - x 2 ) , which can be thought of as anaxiomatization of a theory.

A bootstrap test of, for example,

(1) e = X!-x 2

is carried out by finding computations of each of the quantities in theequation in terms of input and output variables. The computations for thetwo variables on the right hand side of (1) are trivial since they arethemselves input variables. A computation of e is obtained as follows:

where x' = 1 - x. Each step in the computation of e is warranted either bysome Boolean identity or by some consequence of the "theory" describedabove. The equation in measured variables that must be satisfied if thesecomputed values are to confirm (1) is therefore:

and this equation is called the representative of (1) for the computationsgiven. Values of the input variables and of the output variables willconstitute a test of hypothesis (1) if and only if:

since this condition is necessary and sufficient for the computation of e fromthe data to determine a value for e, and since, also, (2) is not a Booleanidentity. Thus, assuming that condition (3) is met by the data, therepresentative (2) of the hypothesis becomes:

(4) z = 0.

Any of the following inputs will therefore test hypothesis (1):

( x j , x2, x.3, x4)(1, 0, 0, 0)(0, 0, 0, 0)(1, 0, 0, 1)(0, 0, 0, 1)

14 Clark Glymour

and the output z must in each case be 0 if the hypothesis is to pass the test, 1if it is to fail the test.

Alternative hypotheses usually considered by computer scientists arethat the e gate is stuck at 1 (s.a. 1.) or that it is stuck at 0 (s.a.O.). The samecomputations apply to either of these hypotheses as to hypothesis (1). Forthe stuck at 1 fault the hypothesis to be tested is e = 1, and therepresentative of the hypothesis for the computations is:

which is, again assuming the condition (3) for the determinacy of e,

(6) z = 1.

As with the first case, we obtain tests of the hypothesis that e is s.a.l. if andonly if the data meet condition (3), and the value of z required for thehypothesis to pass the test is in each case the complement of the valuerequired for hypothesis (1) to pass the tests. Thus each of the inputs is a testof e = 1 if and only if it is a test of (1), and each such test discriminatesbetween the two hypotheses, since the predicted outputs are alwaysdifferent, i.e., the representatives are incompatible.

For the hypothesis that e is s.a.O., everything is as before, but whencondition (3) for the determinacy of e is met, the representative of thehypothesis becomes

(4) z = 0.

Thus no bootstrap test will discriminate between a s.a.O. fault and thenormal condition given by hypothesis (1).

A standard technique in the computer science literature for detectingfacults in logic circuits derives from the observation that, for example, thes.a.O. fault can be discriminated from the normal condition of a line,provided there is a combination of inputs that, in the normal condition,gives the line the value 1, and, for these inputs, the output variable z wouldbe different from the value determined by the normal circuit if the linevalue were 0 instead of 1. This "Boolean derivative" method, applied to thecase discussed above in which the normal hypothesis is (1) and the twoalternative hypotheses are that e is s.a.O. and that e is s.a.l., yields thefollowing necessary and sufficient condition for a set of input values (a1; a2,a3, 34) to be a test that discriminates the normal condition from the s.a.O.fault:


and likewise the necessary and sufficient condition for a discriminationbetween the normal and the s.a.l. fault is:

dzIn equations (7) and (8) -r- is the Boolean derivative, i.e., the Booleande

function whose value is 1 if and only if a change in the value of e changes thevalue of z when x2, x3, x4 are kept constant.

For the case given it can easily be shown that:

2

dzand since e = Xj-x 2 , it follows that e i-j— = 0 identically, and therefore in

decomputer parlance the s.a.O. fault is not testable, i.e., not discriminablefrom the normal circuit element. This is the same result obtained frombootstrap testing. For the s.a.l. fault, the Boolean derivative method yieldsas a condition necessary and sufficient for discriminating the fault:

Thus every input such that x2 = x3 = 0 will discriminate between thenormal case and the s.a. 1. fault. Again this is the same result the bootstrapmethod gives.

The equivalence of the methods is general: Provided a circuit contains noloops, a set of inputs discriminates between a fault and a normal circuitelement according to the bootstrap method if and only if that set of inputsdiscriminates between that fault and the normal circuit element accordingto the Boolean derivative method. (See appendix.)

16 Clark Glymour

6. Probability and ErrorMany philosophers view the analysis of confirmation and testing as a

branch of the theory of probability or of statistics. I do not. I do, however,believe that applications of statistical methods in science and engineeringare often dependent on the structural features of hypothesis testingdescribed in previous sections, and I believe that bootstrap testingprovides one natural setting for certain probabilistic conceptions of thebearing of evidence.

In many cases measurement results are treated as values for randomvariables. Following the customary theory of error, values for the randomvariable E are taken to be normally distributed about the true value of the(nonstochastic) variable e. The mean and variance of the sampling distribu-tion of E are in practice used both for qualitative indications of the value ofthe evidence for some hypothesis about e, and also for more technicalprocedures—hypothesis testing, determination of confidence intervals,etc. The same is true when e and E are not single quantities but a set orvector of quantities and random variables. When some theory T relates thequantities (ei, . . . , e n ) to other quantities xf = f j (e i , . . . ,en), there arecorresponding statistics X, = f j (E] , . . . ,En) that may be used for testinghypotheses about (e\,... ,en) based on T. The very point of the sections on"propagation of error" contained in every statistical manual is to describehow the variances of the E; determine the variances of the XJ5 so as topermit the application of standard statistical testing and estimation tech-niques for hypotheses about the Xj. The conditions of the bootstrap testingschemas are no more than attempts to describe necessary conditions for theapplication of standard statistical hypothesis testing-techniques to hypoth-eses about unmeasured quantities.

7. Bayesian BootstrappingIn scientific contexts, Bayesian probability not only rests on a different

conception of probability from the one embraced by the more "orthodox"tradition in statistics, it also has different applications. Despite my distastefor Bayesian confirmation theory as a general theory of rational belief andrational change of belief—a theory that is, in Glenn Shafer's words, suitedfor "never-never land"—I think Bayesian and related techniques arepractical and appealing in various scientific and engineering circum-stances. In this and the next two sections, we focus on the first of the


conditions in the three schemas for bootstrap testing, and its interactionwith probability.

How can we associate a probability measure, closed under Booleanoperations, with a set of equations or inequations in specified variables?Straightforwardly, by considering for n-variables an n-dimensional spaceSn and associating with every set K of equations or inequations the solutionset of K in Sn. The probability of K is then the value of a probabilitymeasure on Sn for the solution set of K.

Consider a set of variables xf essential to a hypothesis H, and a (prior)probability distribution Prob over the space of the Xj. Let Prob(H) be theprior probability of H, i.e., of the solution set of H. Suppose values xs* arecomputed for the variables \( from the measured variables Cj using sometheory T. Let Tj be the fragment of T used to compute x f , and let (&T) bethe set of Tj for all of the x; occurring in H. We can associate T and &T withsubspaces in the space of the variables x, and e,. Prob and the computedvalues for the variables xf could be taken to determine

(1) Prob(H/Xj - x*) = Prob(H/ej = e* & (&T))

where the e* are the measured values of the Cj.

Where we are dealing with point values xf and where the Tf areequations, (1) will typically require that we conditionalize on a set withprobability measure zero, and so we would need either approximationtechniques or an extended, nonstandard, notion of conditionalization.

Finally, one might take the values e* for e, to confirm H with respect to Tif

(2) Prob(H/Xi = x*) = Prob(H/ej - e* & (&T)) > Prob (H).

This is essentially the proposal made by Aaron Edidin. For Schemas I andII, if we take this procedure strictly, then hypotheses involving the x; thatare not testable because they are entailed by &T will always get posteriorprobability one. On the other hand, if we avoid this difficulty as Edidinsuggests by conditionalizing only when H is not entailed by &T, then

Prob(H/(&T) & e, = e*) will not necessarily be a probability distribution.

8. Jeffrey ConditionalizationSometimes we do have established probabilities for a collection of

hypotheses regarding a set of variables, and we regard a consistent body ofsuch probabilized hypotheses as forming a theory that we wish to test or to

18 Clark Glymour

use to test other hypotheses. Now the Bayesian approach previouslydescribed makes no use of the probabilities of the hypotheses used tocompute values of unmeasured variables from values of measured varia-bles. But it is very natural to let the probabilities of the T; determineprobabilities for the computed values of the xi5 and to use this informationin judging both the posterior probability of any hypothesis H(XJ), as well asthe confirmation of any such hypothesis.

To begin with we shall confine attention to a definite theory T, a set ofvariables x,, and for each xf a single computation x, = T,(ej) derivable fromT. For each such equation (or inequality) there is assumed a probabilityProb(xi) = Prob(xi = Tj(ej)) or Prob(i) for brevity; the joint probabilities forthese hypotheses may or may not be known. Suppose the Cj are measuredand values e* are found. From the computations, values x* = Tj(e*) aredetermined. What confidence do we have that X; = Tj(e*), assuming wehave no doubts about the evidence? We might suppose that

For hypotheses about the xs, how can the probabilistic conclusions, whenindependent, be combined with the prior probabilities of hypotheses togenerate a new, posterior probability? Let Es be set of mutually exclusive,jointly exhaustive propositions, Prob a prior probability distribution andPos(Ej) a new probability for each Ej. Richard Jeffrey's rule for obtaining aposterior distribution from this information is:

(3) Pos(H) = 2 Pos(Ej)-Prob(H/Ei).

Applied to the present context, Jeffrey's rule suggests that we have for anyhypothesis H(xJ

(4) Pos(H) = Prob(&T)-Prob(H/x*) + (l-Prob(&T))-Prob(H/x"*)where x* = Tj(e*) and x* is the set complementary to x* and,e.g., Prob(H/x*) is short for Prob(H/Xi = x* for all Xj in Var (H)).

Equation (4) is not a strict application of Jeffrey's rule (3), however, for (3)uses the posterior probability of the evidence, Pos(Ej), whereas in (4) theprior probability of the computed values, Prob(&T), is used. Carnap, andmore recently H. Field and others, have noted that Jeffrey's Pos(Ej)depends not only on the impact of whatever new evidence there is for E i5

but also on the prior probability of E,. In contrast, the Prob(&T) of (4) neednot depend on the prior distribution of the Xj.


Rule (4) has the following properties:

(i) Evidence E can disconfirm hypothesis H with respect to &T eventhough &T entails H.

(ii) Evidence E cannot confirm hypothesis H with respect to &T if &Tentails H and H entails x{ = Tj(e*) for all i.

(iii) Evidence E cannot confirm hypothesis H with respect to &T if Hentails x; = Tj(e*) for all i and the prior of H is greater than orequal to the prior of &T.

(iv) Evidence E confirms hypothesis H with respect to T if x* entailsH and 1 - Prob (&T) > Prob (H).

"Confirmation" here means the posterior probability is greater than theprior probability.

Results (ii) and (iv) show that the notion of confirmation implicit in (4) isnot very satisfactory, (iv) says, for instance, that even if H is inconsistentwith the computed values of the xi5 H is confirmed provided the probabilityof the theory used is sufficiently low. The reason is that a low probability forthe theory gives a low probability to x*, hence a high probability to x* andthus to hypotheses it entails; (iii), which implies (ii), seems much too stronga feature.

9. Shafer's Belief FunctionsReconsider the interpretation we have given to the relation between the

probability of &T and the number, also equal to the probability of &T,associated with the values of the x, computed from the evidence using &T.The evidence, and the hypotheses &T, and their probability, give usreason to believe that the true value of each x; is Xj*. Prob(&T) measuresthe strength of that reason, on the assumption that the evidence is certain.We have supposed that Prob(&T) is a probability and that it is numericallyequal to the degree of support that e* gives to the xf = x;* hypothesis withrespect to &T. But why should we assume that the degree of support is aprobability in the usual sense? On reflection, we should not. To illustrate,suppose &T is a body of hypotheses relating xf to 6j, which has beenarticulated and investigated and has an established probability about equalto two-thirds. Further suppose that no other comparably detailed hypothe-ses relating Xj to Cj have an established probability. Now let evidence e* beobtained. Using &T and its probability, we infer that e* gives xf = Tj(e*) =xf, a degree of support also equal to two-thirds. Surely that does not mean

20 Clark Glymour

that this same evidence e*, and same theory &T, gives x, = x(* a degree ofsupport equal to one-third. But that conclusion is required by the construalof degree of support as a probability, and is the source of one of thedifficulties with rule (4) of the previous section.

What is required is a way of combining a "degree of support" for a claimwith a prior probability distribution that has no effect on the degree ofsupport that the evidence provides. Jeffrey's rule gives us no way toconstruct the combination effectively. It gives us at best a condition (3),which we might impose on the result of the combination, whatever it is. Atheory proposed by A. Dempster and considerably elaborated in recentyears by G. Shafer1 has most of the desired properties.

To explain how the rudiments of Shafer's theory arise, consider a space ofhypotheses having only two points, h1 and h2. Suppose there is a priorprobability distribution Bel1 over the space ft. Given that 6j = Cj* and thatno other theory of known probability (save extensions of &T) determines h1

or h2 as functions of the e J5 we take the degree of support, Bel2 (h1), thatthe evidence provides for h1 to be

Prob (&T)

if the Xj* satisfy h1. Under these same assumptions, what is the degree ofsupport that the evidence provides for h2? None at all. The evidenceprovides no reason to believe h2, but it does provide some reason tobelieve h1. We set Bel2(h2) = 0. It seems a natural and harmlessconvention to suppose that any piece of evidence gives complete support toany sentence that is certain. Hence Bel2(ft) = 1.

The function Bel2, which represents the degrees of support that 6j*provides with respect to &T, does not satisfy the usual axioms of probabil-ity. Shafer calls such functions Belief functions. They include as specialcases functions such as Bel1, which satisfy the usual axioms of probability.In general, given ft, a belief function Bel is a function 2n^» [0, 1] such thatBel((J>) = 0, Bel (ft) = 1, and for any positive integer n and every collection

Our problem is this: How should Bel1 and Bel2 be used to determine anew belief function in the light of the evidence, 6j*? Shafer's answer isgiven most simply in terms of functions that are determined by the belieffunctions. Given Bel: 2n —> [0, 1], define


for all A C ft.

Shafer calls functions M basic probability assignments. A basic probabilityassignment reciprocally determines a belief function by

for all ACft.

Thus for the unit subsets of ft—{hj and {h2} in our case—Bel (hj) = M(hj).Shafer's version of Dempster's Rule for combining Belief Functions is

where

The combination of Bel1 and Bel2 obtained by so combining M1 and M2 isdenoted Bel = Bel1 © Bel2 where Bel(A) = 2 {M(B;) B1 C A}.

Shafer terms a Belief Function such as Bel2 a simple support function;the associated function M2 is nonzero for a single set other than ft.

Shafer's Belief Functions and Dempster's Rule of combination have thefollowing properties:

(i) Bel1 © Bel2 - Bel2 © Bel1

(ii) If Bel1 is a probability and Bel2 a simple support function, thenBel1 © Bel2 is a probability function.

(iii) If Bel1 is a probability, Bel1 (A) =£ 0, =£ 1, and Bel2 is a simplesupport function with M2(A) =£0, then Bel1 © Bel2(A) > Bel(A).

(iv) Under the same assumptions as in (ii), Bel1 © Bel2 has the form ofJeffrey's rule of conditioning.

(v) Suppose M!(h) = ProbO^) and M2(h) = Prob(T2), where M1

and M2 determine simple support functions and a joint probabil-ity exists for Tj and T2. Then M1 © M2(h) - Prob(T, v T2) if andonly if TI and T2 are probabilistically independent.

In view of these features, the contexts in which it is most plausible to applyShafer's theory seem to be those in which the following conditions obtain:

22 Clark Glymour

(i) The set of mutally incompatible theories T1; . . . ,Tn for which thereare established probabilities do not jointly determine Prob(A/E)but do determine a lower bound for Prob(A/E), where A is a prop-er subset of the space of interest and E is the "direct" evidence.

(ii) Separate pieces of evidence E2, E3 . . . En are statisticallyindependent.

(iii) None of the theories Tl5 . . . , Tn individually entails that theprobability of any proper subset of O is unity.

Condition (iii) is, of course, a version of the conditions characteristic ofbootstrap testing.

10. Application: Probabilistic Fault Detection

The conditions under which the combination of ideas just describedactually obtain are not uncommon in engineering problems. A simpleexample can be found again in fault detection in computer logic circuits.Suppose that by testing circuit elements we know the probability of failureafter n hours of use of a logic gate of any particular kind. Further supposethat the probabilities of failure for individual gates are independent.Finally, suppose that the probability that the gate failure results in a stuck-at-zero fault or a stuck-at-one fault is not known in each case, and theprobability of a nonconstant fault is also not known.

With these assumptions, consider again the fragment of a logic circuitdiscussed in Section 5.


Suppose we are concerned as to whether the gate leading to line e isfunctioning normally, i.e., whether e = Xj • x2. Using the bootstrapconception, we can, for given inputs, compute e as a function of output z.This function is determined by the graph G of all paths in the circuit leadingfrom e to z.

Likewise, from each of the input variables x\,. . . ,xn whose values must befixed at (ai, . . . ,an) in order for e = f(z) to hold, there is a path leading fromthe variable to some node of G. Call the graph of all such paths S.

Assuming that the probabilities of failure of the circuit elements areindependent, we can estimate the probability that e = f(z) for inputs a l5

. . . an from the graphs S and G.We have a prior probability that e = x1-x2 , given by one minus the

probability of failure of the gate leading to the e line. For a measured valuez* of z and inputs (a^. ,an) we also have a probability—that computed in thepreceding paragraph—that e = e* = f(z*). Assuming X! and x2 knownfrom measurement, for appropriate values of the measured variables wethus have a degree of support for Xi*x 2 = e, although we may have nonefor Xi 'X 2 =£ e. If the measured value of z is not the value predicted by thenormal circuit for the measured inputs (including xi and x2) then the sameprocedure may support Xi • x2 =£e. In the more general case, the values ofthe lines leading to the e gate would not be known directly frommeasurement but would also have to be computed, and would for giveninputs have probabilities less than one (determined, obviously, by theprobabilities of the failure of gates in the graph of paths leading/rom inputgates to the e-gate).

24 Clark Glymour

11. Appendix: On the Equivalence of Bootstrap andBoolean Derivative Tests of Circuit Faults

For any logic, circuit without loops and with n input variables and moutput variables, introducing a distinct variable for each line in the circuitgenerates, in a well-known way, a system of equations over a Booleanalgebra. Thus, for any equation so generated, there is a well-defined classof values for input and output variables (i.e., a well-defined collection ofdata sets) that will provide bootstrap tests of that equation with respect tothe system. And, given any equation expressing the line output of a logicgate as a function of the line inputs, there is a determinate class of input andoutput sets that provide bootstrap discriminations of that equation from theequation that puts the gate output equal to 1 (stuck-at-1 fault), and adeterminate class of input and output sets that provide bootstrap discrimi-nations of that equation from the equation which puts the gate output equalto 0 (stuck-at-0 fault).

Given a circuit with n input variables xj and m output variables Zj, let e= e(xjc,. . . ,Xi) be the equation that specifies the value of line e as afunction of the input variables x^, . . . ,x l5 and let zf = Zj(e, xi , . . . ,xn) bethe equation expressing the value of the jth output variable as a function ofe and those input variables from which there is a path to z not passingthrough e. Then a set of inputs (a f , . . . ,an) provides a Boolean derivativediscrimination of e — e(x]<,. . . ,xi) from the hypothesis that e is stuck at 0 ifand only if:

And input values (a j . . . an) provide a Boolean derivative discrimination ofe = e ( X ] < . . . X]) from the hypothesis that e is stuck at 1 if and only if:

In these equations, , for example, signifies the "Boolean h

derivative" of the expression Zj(e, X j . . . xj with respect to e,2 i.e., thefunction of x j . . . xn that is equal to 1 if and only if a change in value of echanges the value of the expression.


The basic idea behind the Boolean derivative method is the natural andcorrect one that a fault is detectable by an input if, were the circuit normal,the value of some output variable would be different for that input than itwould be if the circuit contained the fault in question (but no other faults).

The following equivalence holds: an input (ai . . . an) discriminatesbetween the hypothesis that a circuit line e is "normal" (i.e., as in thecircuit description) and stuck at 1 (0) according to the Boolean derivativemethod if and only if (i) there is a bootstrap test (in the sense of Schema I orIII) with a data set including (a ! . . . an) that tests, with respect to the systemof equations determined by the circuit, an equation E, entailed by thesystem, giving e as a function of input values or of values of lines into thegate of which e is an output line and (ii) there is a bootstrap test with thesame data sets that tests e = 1 (0) with respect to the system of equationsdetermined by the circuit obtained from the original circuit by deleting alllines antecedent to (that is, on the input side of) e, and (iii) the data setconfirms E with respect to the equations of the original circuit if and only ifit disconfirms e — 1 (0) with respect to the equations of the modifiedcircuit.

The equivalence follows from a few observations that may serve as thesketch of a proof. First, if the Boolean derivative condition is met, thensome output variable z( can be expressed as a function of e and inputvariables connected to zf by lines not passing through e. If E, is theequation expressing this relationship, then Ef will hold in both the originaland the modified (i.e., stuck) circuits. Moreover, the Boolean derivativedzj(e,X!,. . . ,xj

; equals 1 exactly for those values of the input variables

which determine e as a function of z,. It follows that every Booleanderivative discrimination is a bootstrap discrimination. Conversely, ifconditions for bootstrap testing are met, there must be an equation

(3) f (e ,xj . . .xn , zi. . .zk) = 0

which constrains e, output variables Z j . . . Z|< and input variables X i . . .xn

connected to one or more of the z\ by lines not passing through e, such thatfor input values ( a ! . . . a,,), equation (3) determines e as a function ofT.I. . .Zk and thus

de

26 Clark Glymour

But each of the zi is a function of the xl. . . xn and e, and thus with Xj = ai;

of e alone:

Now taking the Boolean derivative of equation (5):

where

is equal to the exclusive disjunction

where e' = 1—e, and formula (7) has value 1 if and only if its two componentsubformulae have distinct values. But

g(Zl(e),. . . ,zk(e)) *g(Z l(e ') , . . . ,zk(e '))only if for some i, Zj(e)=£ z,(e'), and hence only if for some i

It follows that there is a Boolean derivative test with inputs (a f. . .an).3

Notes1. G. Shafer, A Mathematical Theory of Evidence (Princeton: Princeton University Press,

1980).2. See for example, A. Thayse and M. Davio, "Boolean Calculus and Its Applications to

Switching Theory," IEEE Trans. Computers, C-22, 1973, and S. Lee, Modern SwitchingTheory and Digital Design (Englewood Cliffs, N.J. : Prentice Hall, 1978).

3. Research for this essay was supported by the National Science Foundation, Grant No.SES 80 2556. I thank Kevin Kelly and Alison Kost for their kind help with illustrations.

i. glymour's bootstrapping theory of …mcps.umn.edu/philosophy/10_1glymour.pdf · ·...

Documents