Is the

Replicability of Experiment

a Principle ofInductive Logic?

John D. NortonDepartment of History and Philosophy of Science

Center for Philosophy of ScienceUniversity of Pittsburgh

1

NO.2

This Talk

3

2. The importance of replicability of experiment in the scientists’ reporting of their methodology.

1. Material theory of induction: a bigger project aimed at understanding the nature of inductive inference in science.

1. + 2. = NO

Why Ask?(if it is a principle)

4

What Powers Inductive Inference?Which are the Good Inductive Inferences?

5

Formal Accounts.Modeled on deductive inference.

Inferences warranted by conformity to universal schema, principles.

Material account.Inductive inferences by facts.There are no universal schema. All induction is local.

vs

This sample of Radium Chloride has

monoclinic crystals just like Barium Chloride

All samples of Radium Chloride have

monoclinic crystals just like Barium Chloride

Enumerative induction.

Some A’s are B.

All A’s are B.

Haüy’s principle.Generally, each crystalline substance has a single characteristic crystallographic form.

Which formal schema, principles work in a formal theory?

6

Bayesian Probability

Enumerative Induction

Inference to the Best Explanation

Hypothetico-Deductive Confirmation

What do the scientists say?

…

Replication is the gold standard

7

“REPLICATION—THE CONFIRMATION OF RESULTS AND CONCLUSIONS FROM ONE STUDY obtained independently in another—is considered the scientific gold standard.”

“The Reproducibility of an experimental result is a fundamental assumption in science.”

8

“The Reproducibility of an experimental result

is a fundamental assumption in science.”

Journal of Irreproducible Results

9

10

Repeatablevs

Reproducible

11

Repeatability vs Reproducibility

12

Repeatabilityrequires that the

same measurementsmust reappear under the

same conditions.

Reproducibilityrequires that the

same measurementmust reappear under

changed conditions.

Repeatability conditions: the same measurement

procedure or test procedure; the same operator;

the same measuring or test equipment used under the

same condition;the same location;

repetition over a short period of time.

Reproducibility conditions: observation conditions where independent test/measurement results are obtained with the same method on identical test/measurement items in different test or measurement facilities with different operators using different equipment.

From: International Organization for Standardization (ISO 21748:2010(E), p. 3)

Replicability

The Formal

Approach

13

A Principle of Replicability?

14

As a principle of inference…

Outcome

replication Veridical

no replicationSpurious

A veridical experimental outcome is one that properly demonstrates the process or effect sought by the experimental design.

A spurious or artefactual experimental outcome fails to do so; it arises from an unintended disruption to the experimental design.

Successful replication of an experiment is a good indicator of a veridical experimental outcome;

failure of replication is a good indicator of a spurious experimental outcome.

Problems

15

Not self-containedWhat is a “…process or effect sought by the experimental design…”

1 “Good indicator”

Inductively incomplete

How good is a single replication?…5 replications? …10 replications? …How good is 5 successful replications vs 3 failures? etc. etc. etc. etc.

2

Defeasible in all ways

successful replication

failed replication

epistemically inert

unreplicated result upheld

3

Case Studies

16

Succesful replication

Failedreplication

Import of replicability upheld (mostly)

Import of replicability discarded

H. Pylori and Stomach Ulcers(result accepted as veridical)

Intercessionary prayer(result rejected as spurious)

Cold fusion(result rejected as spurious; and skeptics discount cases of successful

replication)

Miller experiment contradicts relativity theory(relativity theory upheld)

Material Analysis

17

The Facts that Warrant

18

Inferences concerning outcome of a single experiment warranted by:

Class A. Facts that show outcome can come from process or effect sought.

Class B. Facts that show outcome cannot come from other processes.

Randomization, blinding preclude any other factor producing outcome.

Treatment applied to test group only.

Example:controlled trial

Combine A + B: Outcome does come from process or effect sought.

Replication in a Material Analysis

19

Inferences concerning outcome of a single experiment warranted by:

Class A. Facts that show outcome can come from process or effect sought.

Class B. Facts that show outcome cannot come from other processes.

Denials of import of replication (mostly) due to divergence of views on Class A. facts.

Replicability tests Class B. facts only.

Combine A + B: Outcome does come from process or effect sought.

Cases

20

H. Pylori produce ulcers and gastritis

21

Barry Marshall and Robin WarrenNobel Prize 2005 for work in 1980s

Rapid replication of experiments.

Replication is epistemically inert until a new Class A. fact is accepted:

Bacteria can survive in the highly acidic environment of the stomach.

BUT

Dayton C. Miller’s failed 1925 replications of the Michelson-Morley Experiment

22

Dayton C. Miller’s failed 1925 replications of the Michelson-Morley Experiment

23

I think that the Miller experiments rest on an error in temperature. I have not taken them seriously for a minute.To Michele Besso, December 25, 1926

Spurious outcomes not precluded.

Class B. facts not properly in place.

Failure of replication epistemically inert.

Cold FusionMartin Fleischmann and B. Stanley Pons, March 1989

24

Nuclear fusion reaction created by electrolysis of heavy water in a jar on a laboratory bench.

Mixed success in replication:

5 replications find excess heat.

13 attempts fail.

Report, Energy Research Advisory Board to the US Department of Energy (ERAB), November 1989.

Cold Fusion Skeptics

25

“The initial announcement by Pons and Fleischmann in March 1989 exhibited the discrepancy between heat and fusion products in sharp terms. Namely, the level of neutrons they claimed to observe was 109 times less than that required if their stated heat output were due to fusion.”

“…cold fusion should not be possible based on established theory.”

ERAB, Report.

Class A facts are not hospitable.

All replications are epistemically inert.

Cold Fusion Proponents

26

“If theory and observation are in conflict, theory wins [in the skeptics view].

…reality-based science acknowledges what nature reveals and then attempts to find an explanation. Rejection occurs only if a satisfactory explanation cannot be demonstrated. This demonstration is still in progress for cold fusion. ”

We must find Class A facts that are hospitable.

Replications become epistemically significant.

Sturms, Edmund (2007), The Science of Low Energy Nuclear Reaction: A Comprehensive Compilation of Evidence and Explanations about Cold Fusion. Singapore: World Scientific Publishing. (p. 13)

Controlled Studies of Intercessionary Prayer

27

“The sovereigns are literally the shortest lived of all who have the advantage of affluence. The prayer has therefore no efficacy, unless the very questionable hypothesis be raised, that the conditions of royal life may naturally be yet more fatal, and that their influence is partly, though incompletely,

neutralized by the effects of public prayers.”

Galton (1872)

19th century: a tool of skeptics.

20th century: a tool of believers.

Multiple successful intercessions reported:

Byrd, Randolph C. (1988), “Positive Therapeutic Effects of Intercessory Prayer in a Coronary Care Unit Population,” Southern Medical Journal, 81, pp. 826-29.

Harris, William S. et al. (1999), “A Randomized, Controlled Trial of the Effects of Remote, Intercessory Prayer on Outcomes in Patients Admitted to the Coronary Care Unit,” Archives of Internal Medicine, 159, pp. 2273-78.

etc.

A Catholic, a Protestant and a Jew, set out a test of distant prayer….

28

Chibnall, John T. et al. (2001) “Experiments on Distant Intercessory Prayer: God, Science, and the Lesson of Massah,” Archives of Internal Medicine, 161, pp. 2529-36.

“…became convinced that the very idea of testing distant prayer scientifically was fundamentally unsound.”

“…The epistemology that governs prayer (and all matters of faith) is separate from that which governs nature. Why, then, attempt to explicate it as if it were a controllable, natural phenomenon? “

Class A facts are not hospitable.

All replications are epistemically inert.

Conclusion

29

Superfluity of the Formal Analysis

30

… For the better, since we have not been able to formulate one.

Material analysis gives a transparent account of the the diversity of inductive inferences surrounding replication of experiment.

A formal principle of replicability is superfluous.

The End31