adaptive sampling - helsingin yliopisto - matematiikan ja ... · adaptive sampling: an adaptive...

Adaptive SamplingSteve Thompson

[email protected]

Simon Fraser University

16-17 June 2011

BANOCOSS 2011

Adaptive Sampling – p. 1/??

Adaptive sampling:

An adaptive sampling design is one in which theselection of units to include in the sample depends onvalues of the variable of interest observed during thesurvey.


Sketch

1. Adaptive sampling ideas and examples

2. Design and inference considerations

3. Spatial, network, temporal settings


Rare, clustered population


Random sample of 40 units


Same population


Initial sample of 20 units


Adaptive cluster sample


Changed population!


Initial sample of 20 units


Adaptive cluster sample


Types of sampling designs

The procedure by which we select the sample.

Conventional design : p(s)Procedure for selecting the sample does not depend onvalues of variables of interest observed during the survey.

Adaptive design : p(s | y)Procedure for selecting sample can depend on values ofvariables of interest.

(Design can also depend on auxiliary variables x.)


Approaches to inference from samples

Design based approach:The values of the variables of interest in the population arefixed, unknown constants.

y = (y1, . . . , yN )

Model based approach:The population values are random variables, which we tryto model.

Y1, . . . , YN have some joint probability distributionf(y1, . . . , yN | θ).


Trawl survey, Kodiak Island

L. WatsonAdaptive Sampling – p. 14/??

Optimal sampling strategies

Find the design p(s |y) and estimator Z of populationquantity Z to minimize the mean square error

E(Z − Z)2

subject to unbiasedness, E(Z) = E(Z)

The optimal strategy is in most cases an adaptive one.


Reasoning:

1. Stop part way through the survey and look at what hasbeen observed so far:

initial sample and values (s1, ys1)

2. Choose the rest of the sample s2 to minimize the meansquare error of the estimate given what has been observedso far.

minE[

(Z − Z)2 | s1,ys1

]

(Zacks 1969, Thompson and Seber 1996, Chao and Thompson 2000)


The idea:

Given what you’ve observed so far, choose subsequentsample units with good design conditional on that.

Say the sample is in two phases, s = (s0, s1).The data are d = (s, ys).

E

mins1

E[(τ − τ)2 | s0, ys0 ]︸︷︷︸

using current data

≤ mins

E(τ − τ)2

︸︷︷︸

not using


Practical, efficient designs

Theoretically optimal designs are hard to implement,computationally complex, and overly dependent on modelbased assumptions.

We seek instead practical, efficient, robust designs


Adaptive cluster sampling estimation

y is not unbiased for µ

Unbiased estimate has form

∑ yiαi

αi = network intersection probability,

or Rao-Blackwell form.


Sufficiency, completeness, Rao-Blackwell

sampling data = (s, ys)

sufficient statistic = set of distinct units, associated yvalues

Rao-Blackwell estimate =E[simple estimator | sufficient statistic]

Minimal sufficient statistic is not complete so more thanone possible estimator.


Bering Sea king crab surveyJohnson, Chao, Thompson and Stevens - Draft 10/28/2001

Figure 1.| The result of adaptive cluster sampling during 1995 in the Bering Sea. Initiallyselected primary units were 43, 56 and 86. Neighborhood pattern was north, south, eastand west. Mature female red king crabs (Paralithodes camtschaticus) were sampled witha condition of 18 in the southern stratum, and a condition of 60 in the northern stratum.Maturity was determined by presence of embryos or empty egg cases.{ 15 {


Likelihood function

Prob(data | parameters) = P(s,ys | θ)

L(θ; s,ys) =

∫

p(s |y; θ)f(y; θ)dys

=

∫

(design)(model)d(unobserved)

In general a likelihood function involves both the selectionmechanism (design) and the model and effective inferenceshould take into account both.


“Ignorable” design

If the design depends only on values that are observed andrecorded in the data, then the design disappears fromlikelihood-based estimates.

L(θ; s,ys) = p(s |ys; θ1)

∫

f(y; θ2)dys

But to be ignorable for frequentist model-based inference,the design must be a conventional one p(s), depending onno y-values at all.

It can be argued that in most real situations, the design isignorable for data analysis only if the study used a knownprobability design.


Adaptive sampling in networks


Studies of hidden populations

HIV/AIDS at-risk study

M. Miller


Sampling in networks

Population of units or nodes: 1, 2, . . . , N

Node variables of interest : y1, y2, . . . , yN

Link-indicators or weights: wij , i, j = 1, . . . , N

(Variables of interest associated with pairs of nodes)

Sample : A subset or sequence s of units and pairs of units

from the population: s = (s(1), s(2))y is observed in s(1).w is observed in s(2).


Approaches to inference in network sampling

Design based approach:The values of the variables of interest in the population arefixed, unknown constants.

y = (y1, . . . , yN )

w = {wij}, i, j ∈ {1, . . . , N}

Probability enters only through the design

Model based approach:The population values are random variables, which we tryto model.

Y1, . . . , YN , W11, . . . ,WNN have some joint probabilitydistribution, described by a stochastic graph model


Snowball and Random Walk Designs

1. Snowball designs and inference

2. Random walk designs and inference


Example network population

population graph


Random sample

sample


Snowball sample

sample


One-wave snowball selection probabilities

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

One−wave selection probabilities


The population again

population graph


Random walk sample

walk


Random walk limit selection probabilities

Limit random walk probabilities


Random walk as Markov chain

Wk is the node of the graph selected at kth wave.aij = 1 indicates a link from node i to node j.

{W0,W1,W2, . . . } is a Markov chain withP (Wk+1 = j |Wk = i) = aij/ai·

Q is the transition matrix of the chain,qij = P (Wk+1 = j |Wk = i).

The stationary probabilities (π1, . . . , πN ) satisfy πj =∑

πiqijfor j = 1, . . . , N .


Approach using limiting distribution of random walk

For random walk design with-replacement in asingle-component network and if the links are symmetric ,

then the limiting selection probability is proportional to theperson’s degree (di)

Generalized ratio estimator of mean for behavioralcharacteristic y:

µ =

∑

s yi/di∑

s 1/di


Targeted random walk designs

1. Uniform random walk

2. More general targetting


Targeted walk designs

Let πi(y) denote the desired stationary selection probabilityfor the ith node as a function of its value or degree.

The transition probabilities for the targeted walk are

Pij = qijαij for i 6= j

Pii = 1−∑

j 6=i

Pij

where

αij = min

{πjqjiπiqij

, 1

}


TARGETED RANDOM WALK DESIGNS

1. Random walk as a Markov chain

2. Random, uniform, and targeted walks


Random walk as Markov chain

Wk is the node of the graph selected at kth wave.aij = 1 indicates a link from node i to node j.

{W0,W1,W2, . . . } is a Markov chain withP (Wk+1 = j |Wk = i) = aij/ai·

Q is the transition matrix of the chain,qij = P (Wk+1 = j |Wk = i).

The stationary probabilities (π1, . . . , πN ) satisfy πj =∑

πiqijfor j = 1, . . . , N .


Targeted walk design

Let πi(y) denote the desired stationary selection probabilityfor the ith node as a function of its value or degree.

The transition probabilities for the targeted walk are

Pij = qijαij for i 6= j

Pii = 1−∑

j 6=i

Pij

where

αij = min

{πjqjiπiqij

, 1

}


Uniform targeted walk design

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

population

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Limit selection propabalities

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

1

‘random walk’ sample

2

3

4

5

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

1

uniform walk sample

2

3 4

5


Random, uniform, and targeted walks

0 10 20 30 40 50

0.10

0.20

0.30

0.40

random walk, no jumps

wave

expe

cted

nod

e va

lue

0 10 20 30 40 50

0.10

0.20

0.30

0.40

random walk (with jumps)

wave

expe

cted

nod

e va

lue

0 10 20 30 40 50

0.10

0.20

0.30

0.40

uniform walk

wave

expe

cted

nod

e va

lue

0 10 20 30 40 50

0.10

0.20

0.30

0.40

value 2/1 walk

wave

expe

cted

nod

e va

lue

0 10 20 30 40 50

0.10

0.20

0.30

0.40

degree+1 walk

expe

cted

nod

e va

lue

0 10 20 30 40 50

0.10

0.20

0.30

0.40

degree walk

expe

cted

nod

e va

lue


Adaptive web sampling

1. How they work

2. Variations

3. Inference methods



At any point in the sampling,

• the next unit or set of units is selected from a distributionthat depends on the values of variables of interest in anactive set of units already selected. (follow a link )



At any point in the sampling,

• the next unit or set of units is selected from a distributionthat depends on the values of variables of interest in anactive set of units already selected. (follow a link )

• With some probability, however, the selection may bemade from a distribution not dependent on thosevalues. (random jump )


Population graph

population graph


Adaptive web design

weighted links


Inference

Estimation of a population characteristic such as apopulation mean, degree distribution, or other quantity,based on the sample data.

• Design-basedsimple preliminary estimatorimprove with Rao-Blackwell or resampling


Inference

Estimation of a population characteristic such as apopulation mean, degree distribution, or other quantity,based on the sample data.

• Design-basedsimple preliminary estimatorimprove with Rao-Blackwell or resampling

• Model-basedassume stochastic graph modelproduce realizations from predictive posterior


Design-unbiased estimators

• Start with some preliminary unbiased estimator µ0, suchas the initial sample mean , an unequal probabilityestimator , or conditional probability estimator




• Improve it using the Rao-Blackwell method:

µ = E(µ0|d) =∑

paths

µ0(s)p(s | d)





µ = E(µ0|d) =∑

paths

µ0(s)p(s | d)

d is the minimal sufficient statistic


Estimator based on initial sample mean

• Start with unbiased estimator of µ based on the initialsample s0.

For example,µ01 = y0

or

µ01 =1

N

∑

i∈s0

yiπi


Estimator based on initial sample mean

• Start with unbiased estimator of µ based on the initialsample s0.

For example,µ01 = y0

or

µ01 =1

N

∑

i∈s0

yiπi


µ1 = E(µ01|dr) =∑

{s:r(s)=s}

µ01(s)p(s | dr)


Estimators based on conditional probabilities

τs0 , an unbiased estimator of the population total based on the initialsample s0.

For the kth selection after the initial sample, zk =∑

i∈sckyi + yk/qaki

where qaki is the conditional probability of selecting person i given thecurrent active set ak.

An unbiased estimator of the population mean is

µ02 =1

Nn

[

n0τs0 +

n∑

i=n0+1

zi

]

The improved estimator is

µ2 = E(µ02|dr) =∑

{s:r(s)=s}

µ02(s)p(s | dr)


Composite conditional generalized ratio

N0 an estimator of the population size N based on the initial sample, forexample, N0 =

∑

k∈s0(1/πk).

After the initial sample, Nk = nck + 1/qaki, where nck is the size of thecurrent sample.

A composite estimator of N is

N =1

n

[

n0N0 +

n∑

i=n0+1

Ni

]

A generalized ratio estimator is then formed as the ratio of the twounbiased estimators: µ03 = Nµ02/N

The improved version of this estimator is

µ3 = E(µ03|dr) =∑

{s:r(s)=s}

µ03(s)p(s | dr)


Composite conditional mean of ratios

An alternate way to use the ratios of unbiased estimators in a compositeestimator is

µ04 =1

n

[

n0z0

N0

+

n∑

i=n0+1

zi

Ni

]

The improved version of this estimator is

µ4 = E(µ04|dr) =∑

{s:r(s)=s}

µ04(s)p(s | dr)


Computational issue

µ = E(µ0|dr) =∑

{s:r(s)=s}

µ01(s)p(s)

The sum is over all possible sample paths giving dr.

Sample sequence s = (s0, in0+1, . . . , in) has selection probability

p(s) = p0qan0,i1 · · · qan−1in

For a nonreplacement design, n! reordings of the sample.


Computational issue

µ = E(µ0|dr) =∑

{s:r(s)=s}

µ01(s)p(s)





• n = 9 has 362,880 reordings.


Computational issue

µ = E(µ0|dr) =∑

{s:r(s)=s}

µ01(s)p(s)






• n = 10 has 3.6 million.


Computational issue

µ = E(µ0|dr) =∑

{s:r(s)=s}

µ01(s)p(s)






• n = 10 has 3.6 million.

• n = 20 has 2.4 quintillion (1018), as in “million, billion, trillion,quadrillion, quintillion,...”


Markov chain resampling estimators

Let x be a permutation of the sample s.

The object is to obtain a Markov chain x0, x1, x2, . . . having stationarydistribution p(x | dr).

1. A tentative permutation tk is produced by applying the originalsampling design to the data as if the sample were the wholepopulation.

2. With probability α, tk is accepted and xk = tk, while with probability1− α, tk is rejected and xk = xk−1, where

α = min

{p(tk)

p(xk−1)

pt(xk−1)

pt(tk), 1

}


Spatial adaptive web sampling

spatial population


Network structure of spatial population

population graph


Adaptive web sample

sample


The resulting spatial sample

spatial population


Active set design variations

spatial population

population graph

active set sample active set sample


Blue-winged teal population

spatial population

3 5

24 14

2 3 2

10 103

13639 1

14 122

1772 00000000

00000000

00000

000000

00000000

population graph


Two samples, n=20. Top:n0 = 13. Bottom: n0 = 1.

sample

sample


MSE of estimators depending onn0, with n = 20

5 10 15 20

0.6

0.8

1.0

1.2

1.4

1.6

1.8

n0

mse

µ1µ2µ3µ4


Model-based inference with network designs

(Work with Ove Frank, Mosuk Chow, Mike Kwanisai, andothers).


Bayes predictive inference

Inference about population characteristics based on theBayes predictive posterior distribution given the datad = (s,ys,ws)

P (ys,ws | d) =

∫

P (ys,ws , θ, β | d) dθ dβ

Based on an assumed stochastic graph model f(y,w; θ, β),θ = node paramaters, β = link parameters.


Sampling from predictive posterior

The object is to produce many realizations of the entirepopulation from the posterior distribution given the sampledata.

This is the data augmentation step of a Markov chain MonteCarlo procedure.


Within Bayes: The likelihood function

The likelihood function depends on both the design used inobtaining the data and the model describing the population.

Prob(data | parameters) = P(s,ys,ws | θ, β)

L(θ, β; s,ys,ws) =

∫

p(s |y,w; θ, β)f(y,w; θ, β)dysdws

=

∫

(design)(model)d(unobserved)


MCMC for network Bayes inference

1. Using current values of θ and β, select a realization of(ys,ws) from P (ys,ws | θ, β, s,ys,ws).

2. Using the values (ys,ws) obtained in step (1) toaugment the data values (ys,ws), select new parametervalues (θ, β) from the posterior distribution of theparameters given the whole graph realizationπ(θ, β |ys,ys,ws,ws)

Repeat.


Bayes predictive inference; Actual pattern

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2

actual pattern in region


Sample and observed values

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1[ss]

co

ord

2[s

s]

sample and observed values


Realization from posterior distribution

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2

inferred possible pattern, given data



0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2



Median of posterior distribution

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2

median of possible patterns, given data


Systematic sample, 16 sites

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

x

exp

((−

(x/d

elta

)^2

))

spatial covariance

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1[ss]

co

ord

2[s

s]

sample data

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2

conditional realization

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2

conditional median


Systematic sample, 4 sites

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

x

exp

((−

(x/d

elta

)^2

))

spatial covariance

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1[ss]

co

ord

2[s

s]

sample data

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2

conditional median


Random sample, 16 sites

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

x

exp

((−

(x/d

elta

)^2

))

spatial covariance

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1[ss]

co

ord

2[s

s]

sample data

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2

conditional median


MCMC data augmentation steps

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1[ss]

co

ord

2[s

s]

sample data

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2

mcmc augmented data 1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

coord1

co

ord

2



Design and estimation comparisons

population graph


Random walk n=20, initial pp-degree

sample mean, random walk

rwmean

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

mean of draws, random walk

rwnommean

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4

gen ratio est, random walk

rwnaive

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4

gen ratio of draws, random walk

rwnaivenom

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4


5 random walks, n=4 each, pp-deg starts


rwmean

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

2.5


rwnommean

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

2.5


rwnaive

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0


rwnaivenom

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0


random walk, n=20 , equal probability start


rwmean

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45


rwnommean

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

6


rwnaive

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45


rwnaivenom

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

6


AWS, n0=1, n=20, random links, jump=.1

generalized ratio estimate 1

dgre1

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4


dgre2

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4


AWS, n0=10, n=20, random links, jump=.1


dgre1

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4


dgre2

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4


Design and model based estimators, AWS n0=10, n=20

sample mean

ybar

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

rb initial mean

rbw0vec

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

3.0

rb norepl est

rbnoreplvec

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

3.0

rb nr alt

rbnraltvec

De

nsity

0.0 0.2 0.4 0.6 0.8 1.00

.01

.02

.03

.0

rb nr alt0

rbnraltvec0

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

bayes predictor

bayes.predtvec

De

nsity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

3.0


Designs and Estimators

grhh grht gre est1 est3 gre est1 est3 bayes

0.0

00

.02

0.0

40

.06

0.0

80

.10

0.1

20

.14

DESIGN

Random WalkAWS n0=1, n=20AWS n0=10, n=20


Empirical Example

HIV/AIDS at-risk hidden population: Colorado SpringsStudy on the heterosexual transmission of HIV/AIDS(Potterat et al. 1993, Rothenberg et al. 1995, Darrow et al.1999)


Colorado springs study population

population graph


Sample of 80 individuals

Initial n0 = 10, final n = 20, m = 4 independent selections.

samplesamplesamplesample


Design and Model based inferences

sample mean

s.mean

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

67

mle

mle.theta

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

02

46

bayes estimator

bayes.t

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

67

bayes predictor

pred.tD

ensi

ty

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

67

design unbiased

dunbiased

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

02

46

design consistent

dconst

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

02

46

(Sex worker data) Adaptive Sampling – p. 95/??

HIV Behavioral Monitoring Design Study

population graph


Design-based and Bayes estimators

sample mean

ybar

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

rb initial mean

rbw0vec

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

3.0

rb norepl est

rbnoreplvec

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

3.0

rb nr alt

rbnraltvec

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

3.0

rb nr alt0

rbnraltvec0

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

bayes predictor

bayes.predtvec

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

1.0

2.0

3.0


Empirical Example

HIV/AIDS at-risk hidden population: Colorado SpringsStudy on the heterosexual transmission of HIV/AIDS(Potterat et al. 1993, Rothenberg et al. 1995, Darrow et al.1999)


Colorado springs study population

population graph


Sample of 80 individuals

Initial n0 = 10, final n = 20, m = 4 independent selections.

samplesamplesamplesample


Estimating idu use, random links design

initial mean

y0

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0 1.2

02

4rb initial mean

rbw0vec

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0 1.2

02

4

norepl est

norepl.est

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0 1.2

0.0

1.0

2.0

rb norepl est

rbnoreplvec

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0 1.2

0.0

1.0

2.0

norepl alt

noreplalt

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0 1.2

0.0

1.5

3.0

rb nr alt

rbnraltvecD

ensi

ty

0.0 0.2 0.4 0.6 0.8 1.0 1.2

02

4

norepl alt0

noreplalt0

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0 1.2

02

4

rb nr alt0

rbnraltvec0

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0 1.2

02

4


Estimating idu use, weighted links design

initial mean

y0

Den

sity

0.0 0.5 1.0 1.5

02

4rb initial mean

rbw0vec

Den

sity

0.0 0.5 1.0 1.5

02

4

norepl est

norepl.est

Den

sity

0.0 0.5 1.0 1.5

0.0

1.0

2.0

rb norepl est

rbnoreplvec

Den

sity

0.0 0.5 1.0 1.5

0.0

1.0

2.0

norepl alt

noreplalt

Den

sity

0.0 0.5 1.0 1.5

01

23

rb nr alt

rbnraltvecD

ensi

ty

0.0 0.5 1.0 1.5

01

23

4

norepl alt0

noreplalt0

Den

sity

0.0 0.5 1.0 1.5

02

46

rb nr alt0

rbnraltvec0

Den

sity

0.0 0.5 1.0 1.5

02

46


Degree distribution, HIV/AIDS study

0 5 10 15 20

0.0

0.1

0.2

0.3

0.4

degree distribution

degree

freq

uenc

y

0.0 0.5 1.0 1.5 2.0 2.5 3.0

−6

−5

−4

−3

−2

−1

log(degree)

log(

freq

)

0 5 10 15 20

0.0

0.1

0.2

0.3

0.4

sample degree distribution

degree

freq

uenc

y

0.0 0.5 1.0 1.5 2.0 2.5 3.0

−6

−5

−4

−3

−2

−1

log(degree)

log(

freq

)


Estimating mean degree

average degree in sample

degree

De

nsity

0 2 4 6 8

0.0

0.1

0.2

0.3

0.4

estimate of mean degree

rbw0vec

De

nsity

0 2 4 6 80

.00

.10

.20

.30

.40

.50

.6


Design and Model based inferences

sample mean

s.mean

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

67

mle

mle.theta

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

02

46

bayes estimator

bayes.t

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

67

bayes predictor

pred.tD

ensi

ty

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

67

design unbiased

dunbiased

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

02

46

design consistent

dconst

Den

sity

0.0 0.2 0.4 0.6 0.8 1.0

02

46

(Sex worker data) Adaptive Sampling – p. 105/??

Spatial-Temporal designs

For detecting releases of biological pathogens and otherairborne health hazards,

is it better to set out sensors in fixed positions or to havethem move in some pattern?


The more basic sampling question:

What is the best design for sampling a population that ischanging, when the sampling units themselves may moveas observations are collected.


Background

Early health warning of exposure to airborne biologicalpathogens

Bionet Project places fixed sensor units in selected cities

Builds on the Envionmental Protection Agency’s air qualitymonitoring program


Purpose:

• Rapid health response - earlier diagnosis and treatment


Purpose:

• Rapid health response - earlier diagnosis and treatment

• Environmental remediation


“Streets and avenues design”


Array of rectangular (square) paths


Sample size, moving units

4 6 8 10 12 14 16

0.70

0.75

0.80

0.85

0.90

0.95

1.00

n

prob

det

ect

streets

squareslines

one line


Sample size, fixed units

10 20 30 40 50 60

0.2

0.4

0.6

0.8

1.0

n

prob

det

ect

fixed

moving


Adaptive designs in space-time-network settings


adaptive sampling - helsingin yliopisto - matematiikan ja ... · adaptive sampling: an adaptive...

Documents