prior sensitivity and model choice · dic1 = 271 dic2 = 98 dic1 - dic2 = 172 conclude: overwhelming...

Sensitivity DIC

Prior sensitivity and model choice

Alex Cook

Week 9

Alex Cook, ST5219, Bayesian Hierarchical Modelling 1/30

Sensitivity DIC

HIV trial

x = 51

n = 8197

x ∼ Bin(n, p)

p ∼ Be(1, 1)

E(p) = 0.5

V(p) ≈ 0.292

p

dens

ity

0% 1% 2%


Sensitivity DIC

HIV trial

x = 51

n = 8197

x ∼ Bin(n, p)

p ∼ Be(1, 97)

E(p) = 0.01

V(p) = 0.012

p

dens

ity

0% 1% 2%


Sensitivity DIC

HIV trial

x = 51

n = 8197

x ∼ Bin(n, p)

p ∼ Be(90, 809)

E(p) = 0.1

V(p) = 0.012

p

dens

ity

0% 1% 2%


Sensitivity DIC

HIV trial

x = 51

n = 8197

x ∼ Bin(n, p)

p ∼ Be(90, 809)

E(p) = 0.1

V(p) = 0.012

p

dens

ity

0% 10% 20%


Sensitivity DIC

Take home

Lesson learnt

Informative priors can have big effects on posteriors


Sensitivity DIC

Simulated Poisson data

set.seed(54321)

n=100

lambda=10

x=rpois(n,lambda)

dlambda=0.01

lambda vector

=seq(1,20,dlambda)

x

Fre

quen

cy0 10 20

010

20


Sensitivity DIC


> prior1=dunif(lambda vector,0,100)

> logprior1=log(prior1)

> a=logprior1

> for(i in 1:n)a=a+

dpois(x[i],lambda vector,log=TRUE)

> a=exp(a-max(a));posterior1=a(sum(a)*dlambda)

> plot(lambda vector,posterior1,type=’l’,

xlab=expression(lambda),ylab=’density’)

> lines(lambda vector,prior1,lty=2)


Sensitivity DIC


prior1=dunif(lambda vector,0,100)

0 5 10 15 20

λ

dens

ity

0.0

0.5

1.0


Sensitivity DIC


prior2=dexp(lambda vector,0.01)

0 5 10 15 20

λ

dens

ity

0.0

0.5

1.0


Sensitivity DIC


prior3=dnorm(lambda vector,0,100)

0 5 10 15 20

λ

dens

ity

0.0

0.5

1.0


Sensitivity DIC

Take home

Lesson learnt

Informative priors can have big effects on posteriors

Non-informative priors: form doesn’t matter much


Sensitivity DIC

Classical model selection

yi ∼ N(µi , σ2)

M1 : µi = a + bxi

M2 : µi = a + bxi + cx2i

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●● ●

●●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●●

●●

●

●

●●

●

●

● ●

●

●

●

●

●

●

2 3 4 5 6

100

110

120

130

140

x

y

You were probably taught. . .

NHST of H0 : c = 0, if p < α choose M2


Sensitivity DIC


yi ∼ N(µi , σ2)

M1 : µi = a + bxi

M2 : µi = a + c exp(xi)

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●● ●

●●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●●

●●

●

●

●●

●

●

● ●

●

●

●

●

●

●

2 3 4 5 6

100

110

120

130

140

x

y

What to do now?

NHST of H0 :?


Sensitivity DIC



Sensitivity DIC

Bayesian model selection

DIC:

Deviance: D(θ) = −2 log f (data|θ)

Posterior mean deviance: D̄ =∫

D(θ)f (θ|data) dθ

Effective # parameters: pD = D̄ − D(θ̄)

DIC: DIC = D̄ + pD

Spiegelhalter et al (2002, J Roy Stat Soc ser B64:583–639) recommend same scale as AIC


Sensitivity DIC

Radish data

days post sowing

% in

fect

ed

0 7 14 21

0

20

40

60

80

100


Sensitivity DIC

Radish data

Model 1: no difference model

Xi ∼ Bin(ni , p)

p ∼ Be(1, 1)


Sensitivity DIC

Radish data

Model 2: hierarchical model

Xi ∼ Bin(ni , pi)

pi ∼ Be(a, b)

a ∼ exp(0.0001)

b ∼ exp(0.0001)


Sensitivity DIC

Calculating the DIC

logposterior1=function(current,data){

current$loglikelihood =sum(dbinom(data$I,data$N,current$p,log=TRUE))

current$D = -2*current$loglikelihoodcurrent$logprior = dbeta(current$p,1,1,log=TRUE)current$logposterior = current$logprior +

current$loglikelihoodcurrent

}


Sensitivity DIC

Calculating the DIC

phat=mean(dump$p)current$p=phatcurrent=logposterior1(current,data)Dbar1=mean(dump$D)pD1=Dbar1-current$D

DIC1=Dbar1+pD1


Sensitivity DIC

Calculating the DIC

logposterior2=function(current,data){

current$loglikelihood =sum(dbinom(data$I,data$N,current$p,log=TRUE))

current$D = -2*current$loglikelihoodcurrent$logprior =

sum(dbeta(current$p,current$a,current$b,log=TRUE))+dexp(current$a,0.001,log=TRUE)dexp(current$b,0.001,log=TRUE)

current$logposterior = current$logprior +current$loglikelihood

current

}


Sensitivity DIC

Calculating the DIC

phat=c();for(i in 1:13)phat[i]=mean(dump$p[,i])current$p=phatcurrent=logposterior2(current,data)Dbar2=mean(dump$D)pD2=Dbar2-current$D

DIC2=Dbar2+pD2


Sensitivity DIC

Fitted models

0.70 0.75 0.80 0.85 0.90 0.95 1.00

020

4060

8010

0

probability of disease

dens

ity


Sensitivity DIC

Fitted models

100 150 200 250

0.0

0.2

0.4

0.6

0.8

deviance

dens

ity


Sensitivity DIC

Comparing the DIC

DIC1 = 271

DIC2 = 98

DIC1 - DIC2 = 172

Conclude:

Overwhelming evidence against homogeneous diseaserates


Sensitivity DIC

Better Bayesian model selection


prior sensitivity and model choice · dic1 = 271 dic2 = 98 dic1 - dic2 = 172 conclude: overwhelming...

Documents