all about that bayes: probability, statistics, and the quest to quantify uncertainty

LLNL-PRES-697098 This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC

All About That Bayes Probability, Sta4s4cs, and the Quest to Quan4fy Uncertainty

Kris%n P. Lennox Director of Sta%s%cal Consul%ng July 28, 2016

2 LLNL-PRES-697098

Man of the (Literal) Hour

Probably not Thomas Bayes, but often mistaken for him Source: Wikipedia

3 LLNL-PRES-697098

Central Dogma of Inferen4al Sta4s4cs

Statisticians use probability to describe uncertainty.

4 LLNL-PRES-697098

You Are Here

Why This Matters

What is Probability?

What is Uncertainty?

An Incomplete History of Uncertainty Quantification

The BIG Reveal

5 LLNL-PRES-697098

You Are Here

Why This Matters




The BIG Reveal

6 LLNL-PRES-697098

§  Probability

§  Distribu%on

§  Parameter

§  Likelihood


1933

A. N. Kolmogorov Copyright MFO, Creative Commons License

7 LLNL-PRES-697098

A. N. Kolmogorov Copyright MFO, Creative Commons License

§  Probability is a measure.

§  Distribu%on

§  Parameter

§  Likelihood


1933

8 LLNL-PRES-697098


§  Distribu%ons define measure of events.

§  Parameter

§  Likelihood Exponential Normal/Gaussian


9 LLNL-PRES-697098



§  Parameters define distribu%ons.

§  Likelihood Exponential Normal/Gaussian


10 LLNL-PRES-697098

f (x) = Pr(X = x |Θ =θ )




§  Likelihood fixes data and varies parameters.


11 LLNL-PRES-697098




§  Likelihood fixes data and varies parameters.

l(θ ) = Pr(X = x |Θ =θ )


12 LLNL-PRES-697098

You Are Here

Why This Matters




The BIG Reveal

13 LLNL-PRES-697098

A Fable The Sta4s4cal Lunch Bunch and the Summer Student Revolt of ‘15

1

14 LLNL-PRES-697098


2

15 LLNL-PRES-697098


16 LLNL-PRES-697098


17 LLNL-PRES-697098


18 LLNL-PRES-697098


3

19 LLNL-PRES-697098


20 LLNL-PRES-697098


21 LLNL-PRES-697098


22 LLNL-PRES-697098


4

23 LLNL-PRES-697098


24 LLNL-PRES-697098

You Are Here

Why This Matters




The BIG Reveal

25 LLNL-PRES-697098

In the Beginning…

1654

P. Fermat Source: Wikipedia, Creative Commons License

B. Pascal Source: Wikipedia, Creative Commons License

26 LLNL-PRES-697098

Thomas Bayes and the Doctrine of Chances

1763

Still not Thomas Bayes Source: Wikipedia

27 LLNL-PRES-697098

Blindfolded 1-‐Dimensional Table Bocce

28 LLNL-PRES-697098


Prio

r Den

sity

n=0

29 LLNL-PRES-697098


Pos

terio

r Den

sity

n=1

30 LLNL-PRES-697098

Pos

terio

r Den

sity


n=2

31 LLNL-PRES-697098

Pos

terio

r Den

sity


n=10

32 LLNL-PRES-697098

Pos

terio

r Den

sity


n=25

33 LLNL-PRES-697098

§ What is the probability that event x occurs given that event y occurs?

Bayes Theorem

Pr(X = x |Y = y) = Pr(Y = y | X = x)Pr(X = x)Pr(Y = y)

34 LLNL-PRES-697098

What is the probability that our distribu,on parameter is θ given that we have observed data x?

Bayes Theorem – Bayesian Version

Pr(Θ =θ | X = x) = Pr(X = x |Θ =θ )Pr(Θ =θ )Pr(X = x)

Prior distribution of θ Posterior distribution of θ given x

A constant of integration that most people don’t talk about

Likelihood

35 LLNL-PRES-697098

The Man Who Invented Sta4s4cs

18th Century

P. S. Laplace Source: Wikipedia, Creative Commons License

36 LLNL-PRES-697098

The Prior and Its Enemies

Pr(Θ =θ | X = x)∝Pr(X = x |Θ =θ )Pr(Θ =θ )Posterior distribution of θ given x

Likelihood

Where does the prior come from?

Prior distribution of θ

37 LLNL-PRES-697098

Pr(Θ =θ | X = x)∝Pr(X = x |Θ =θ )Pr(Θ =θ )Prior distribution of θ

Posterior distribution of θ given x

Likelihood

What is the probability that the sun will rise tomorrow?

The Sun Will Come Out Tomorrow?

38 LLNL-PRES-697098

Pr(Θ =θ | X = x)∝Pr(X = x |Θ =θ )Posterior distribution of θ given x

Likelihood



in (0,1)

39 LLNL-PRES-697098



E θ | X = x[ ] = # of times sun has come up + 1# of times sun has come up + 2

40 LLNL-PRES-697098

The Sun Will Come Out Tomorrow?*


= 0.9999995=5000×365.25 + 1 5000×365.25 + 2

*As calculated by Laplace

41 LLNL-PRES-697098

The Frequen4sts

Early 20th Century

J. Neyman Source: Wikipedia, Creative Commons License

R. A. Fisher Source: Wikipedia, Creative Commons License

42 LLNL-PRES-697098

Case Study: Interval Es4ma4on

What is lowest reasonable Pr(X=1)?

43 LLNL-PRES-697098


Bayesian Solu%on (Credible Interval)

1.  Pick a prior.

44 LLNL-PRES-697098



1.  Pick a prior.

2.  Calculate posterior.

45 LLNL-PRES-697098


1.  Pick a prior.

2.  Calculate posterior.

3.  Find 5th percen%le.


95% (subjec%ve) probability that Pr(X=1) is at least 15.3%.

46 LLNL-PRES-697098


Frequen%st Solu%on (Confidence Interval)

1.  Determine all results that are as or more consistent with outcome of interest.

Care about 12 or more 1s in 50 rolls.

47 LLNL-PRES-697098

Frequen%st Solu%on (Confidence Interval)

1.  Determine all results that are as or more consistent with outcome of interest.

2.  Iden%fy all Pr(X=1) that have >5% chance of producing 12 or more 1s.

With 95% confidence, Pr(X=1) is at least 14.5%.


Pr(

12 o

r mor

e 1s

in 5

0 ro

lls)

Pr(X=1)

48 LLNL-PRES-697098


Both give Pr(X=1) >15%*

with “uncertainty” of 5%.

*15.3% (Bayesian with Jeffreys prior) vs. 14.5% (frequen%st)

… but confidence interval takes a lot longer to explain.

49 LLNL-PRES-697098

Why Did This Catch On?

Objec%ve probability is restric%ve, but results mean the same thing to everyone.

(Even if you don’t know what they mean.)

50 LLNL-PRES-697098

BaXle of the Bayesians

20th Century -‐ ???

vs

Representative likeness of a subjective Bayesian Representative likeness of an objective Bayesian

51 LLNL-PRES-697098

The Search For Scorpion

1968

52 LLNL-PRES-697098


1968

Contact M8/3

53 LLNL-PRES-697098


1968

Scorpion location

54 LLNL-PRES-697098

Computa4on

Late 20th Century

55 LLNL-PRES-697098

You Are Here

Why This Matters




The BIG Reveal

56 LLNL-PRES-697098

My Uncertainty Quan4fica4on Toolbox

57 LLNL-PRES-697098

What Have We Learned Today?

Statisticians use probability to describe uncertainty.

We do not always agree about

how this should be done.

58 LLNL-PRES-697098

Ques%ons? [email protected]

Thank you for coming!

Thomas Bayes would never have worn these glasses.

59 LLNL-PRES-697098

§  The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy. McGrayne, S. B. Yale University Press. (2011)

§  Bruno de FineJ: Radical Probabilist. Ed. Galavoj, M. C. Texts in Philosophy 8. College Publica%ons. (2009)

§  Fisher, Neyman, and the Crea,on of Classical Sta,s,cs. Lehmann, E. L. Springer. (2011)

§  The History of Probability and Sta,s,cs and Their Applica,ons Before 1750. Hald, A. Wiley-‐Interscience. (2003)

§  Founda,ons of the Theory of Probability. Kolmogorov, A. N. 2nd English Edi%on. Chelsea Publishing Company. (1950)

§  “Opera%ons Analysis During the Underwater Search for Scorpion.” Richardson, H. R. and Stone, L. D. Naval Research Quarterly. 18, pp. 141-‐157 (1971)

§  “An Essay Towards Solving a Problem in the Doctrine of Chances.” Bayes, T. and Price, R. Philosophical Transac,ons. 53, pp. 370-‐418 (1763)

§  “Sta%s%cal Analysis and the Illusion of Objec%vity.” Berger, J. and Berry, D. American Scien,st. 76, pp. 159-‐165 (1988)

§  “You May Believe You are a Bayesian but You are Probably Wrong.” Senn, S. Ra,onality, Markets and Morals. 2, pp.48-‐66 (2011)

§  “The Case for Objec%ve Bayes.” Berger, J. Bayesian Analysis. 1, pp. 1-‐17 (2004)

§  “When Genius Errs: R.A. Fisher and the Lung Cancer Controversy.” Stoley, P. D. American Journal of Epidemiology. 133, pp. 416-‐425 (1991)

§  “The Evolu%on of Markov Chain Monte Carlo Methods.” Richey, M. The American Mathema,cal Monthly. 117, 383-‐413 (May 2010)

§  and of course Wikipedia.org

Further Reading

all about that bayes: probability, statistics, and the quest to quantify uncertainty

Science