Is the
Replicability of Experiment
a Principle ofInductive Logic?
John D. NortonDepartment of History and Philosophy of Science
Center for Philosophy of ScienceUniversity of Pittsburgh
1
NO.2
This Talk
3
2. The importance of replicability of experiment in the scientists’ reporting of their methodology.
1. Material theory of induction: a bigger project aimed at understanding the nature of inductive inference in science.
1. + 2. = NO
Why Ask?(if it is a principle)
4
What Powers Inductive Inference?Which are the Good Inductive Inferences?
5
Formal Accounts.Modeled on deductive inference.
Inferences warranted by conformity to universal schema, principles.
Material account.Inductive inferences by facts.There are no universal schema. All induction is local.
vs
This sample of Radium Chloride has
monoclinic crystals just like Barium Chloride
All samples of Radium Chloride have
monoclinic crystals just like Barium Chloride
Enumerative induction.
Some A’s are B.
All A’s are B.
Haüy’s principle.Generally, each crystalline substance has a single characteristic crystallographic form.
Which formal schema, principles work in a formal theory?
6
Bayesian Probability
Enumerative Induction
Inference to the Best Explanation
Hypothetico-Deductive Confirmation
What do the scientists say?
…
Replication is the gold standard
7
“REPLICATION—THE CONFIRMATION OF RESULTS AND CONCLUSIONS FROM ONE STUDY obtained independently in another—is considered the scientific gold standard.”
“The Reproducibility of an experimental result is a fundamental assumption in science.”
8
“The Reproducibility of an experimental result
is a fundamental assumption in science.”
Journal of Irreproducible Results
9
10
Repeatablevs
Reproducible
11
Repeatability vs Reproducibility
12
Repeatabilityrequires that the
same measurementsmust reappear under the
same conditions.
Reproducibilityrequires that the
same measurementmust reappear under
changed conditions.
Repeatability conditions: the same measurement
procedure or test procedure; the same operator;
the same measuring or test equipment used under the
same condition;the same location;
repetition over a short period of time.
Reproducibility conditions: observation conditions where independent test/measurement results are obtained with the same method on identical test/measurement items in different test or measurement facilities with different operators using different equipment.
From: International Organization for Standardization (ISO 21748:2010(E), p. 3)
Replicability
The Formal
Approach
13
A Principle of Replicability?
14
As a principle of inference…
Outcome
replication Veridical
no replicationSpurious
A veridical experimental outcome is one that properly demonstrates the process or effect sought by the experimental design.
A spurious or artefactual experimental outcome fails to do so; it arises from an unintended disruption to the experimental design.
Successful replication of an experiment is a good indicator of a veridical experimental outcome;
failure of replication is a good indicator of a spurious experimental outcome.
Problems
15
Not self-containedWhat is a “…process or effect sought by the experimental design…”
1 “Good indicator”
Inductively incomplete
How good is a single replication?…5 replications? …10 replications? …How good is 5 successful replications vs 3 failures? etc. etc. etc. etc.
2
Defeasible in all ways
successful replication
failed replication
epistemically inert
unreplicated result upheld
3
Case Studies
16
Succesful replication
Failedreplication
Import of replicability upheld (mostly)
Import of replicability discarded
H. Pylori and Stomach Ulcers(result accepted as veridical)
Intercessionary prayer(result rejected as spurious)
Cold fusion(result rejected as spurious; and skeptics discount cases of successful
replication)
Miller experiment contradicts relativity theory(relativity theory upheld)
Material Analysis
17
The Facts that Warrant
18
Inferences concerning outcome of a single experiment warranted by:
Class A. Facts that show outcome can come from process or effect sought.
Class B. Facts that show outcome cannot come from other processes.
Randomization, blinding preclude any other factor producing outcome.
Treatment applied to test group only.
Example:controlled trial
Combine A + B: Outcome does come from process or effect sought.
Replication in a Material Analysis
19
Inferences concerning outcome of a single experiment warranted by:
Class A. Facts that show outcome can come from process or effect sought.
Class B. Facts that show outcome cannot come from other processes.
Denials of import of replication (mostly) due to divergence of views on Class A. facts.
Replicability tests Class B. facts only.
Combine A + B: Outcome does come from process or effect sought.
Cases
20
H. Pylori produce ulcers and gastritis
21
Barry Marshall and Robin WarrenNobel Prize 2005 for work in 1980s
Rapid replication of experiments.
Replication is epistemically inert until a new Class A. fact is accepted:
Bacteria can survive in the highly acidic environment of the stomach.
BUT
Dayton C. Miller’s failed 1925 replications of the Michelson-Morley Experiment
22
Dayton C. Miller’s failed 1925 replications of the Michelson-Morley Experiment
23
I think that the Miller experiments rest on an error in temperature. I have not taken them seriously for a minute.To Michele Besso, December 25, 1926
Spurious outcomes not precluded.
Class B. facts not properly in place.
Failure of replication epistemically inert.
Cold FusionMartin Fleischmann and B. Stanley Pons, March 1989
24
Nuclear fusion reaction created by electrolysis of heavy water in a jar on a laboratory bench.
Mixed success in replication:
5 replications find excess heat.
13 attempts fail.
Report, Energy Research Advisory Board to the US Department of Energy (ERAB), November 1989.
Cold Fusion Skeptics
25
“The initial announcement by Pons and Fleischmann in March 1989 exhibited the discrepancy between heat and fusion products in sharp terms. Namely, the level of neutrons they claimed to observe was 109 times less than that required if their stated heat output were due to fusion.”
“…cold fusion should not be possible based on established theory.”
ERAB, Report.
Class A facts are not hospitable.
All replications are epistemically inert.
Cold Fusion Proponents
26
“If theory and observation are in conflict, theory wins [in the skeptics view].
…reality-based science acknowledges what nature reveals and then attempts to find an explanation. Rejection occurs only if a satisfactory explanation cannot be demonstrated. This demonstration is still in progress for cold fusion. ”
We must find Class A facts that are hospitable.
Replications become epistemically significant.
Sturms, Edmund (2007), The Science of Low Energy Nuclear Reaction: A Comprehensive Compilation of Evidence and Explanations about Cold Fusion. Singapore: World Scientific Publishing. (p. 13)
Controlled Studies of Intercessionary Prayer
27
“The sovereigns are literally the shortest lived of all who have the advantage of affluence. The prayer has therefore no efficacy, unless the very questionable hypothesis be raised, that the conditions of royal life may naturally be yet more fatal, and that their influence is partly, though incompletely,
neutralized by the effects of public prayers.”
Galton (1872)
19th century: a tool of skeptics.
20th century: a tool of believers.
Multiple successful intercessions reported:
Byrd, Randolph C. (1988), “Positive Therapeutic Effects of Intercessory Prayer in a Coronary Care Unit Population,” Southern Medical Journal, 81, pp. 826-29.
Harris, William S. et al. (1999), “A Randomized, Controlled Trial of the Effects of Remote, Intercessory Prayer on Outcomes in Patients Admitted to the Coronary Care Unit,” Archives of Internal Medicine, 159, pp. 2273-78.
etc.
A Catholic, a Protestant and a Jew, set out a test of distant prayer….
28
Chibnall, John T. et al. (2001) “Experiments on Distant Intercessory Prayer: God, Science, and the Lesson of Massah,” Archives of Internal Medicine, 161, pp. 2529-36.
“…became convinced that the very idea of testing distant prayer scientifically was fundamentally unsound.”
“…The epistemology that governs prayer (and all matters of faith) is separate from that which governs nature. Why, then, attempt to explicate it as if it were a controllable, natural phenomenon? “
Class A facts are not hospitable.
All replications are epistemically inert.
Conclusion
29
Superfluity of the Formal Analysis
30
… For the better, since we have not been able to formulate one.
Material analysis gives a transparent account of the the diversity of inductive inferences surrounding replication of experiment.
A formal principle of replicability is superfluous.
The End31