Sampling and Allocation

Definitions

• Population - the group you want to talk about• Target population - the group that can be sampled• Sampling - the process of going from the target population to

the sample• Sample - those you have in your study• Allocation - how you split people into conditions• Measurements - what and how you measure things

Gold Standards

• For sampling– Simple random sampling (SRS)

• For allocation – Random allocation

The “Gold Standard” for Sampling

- Simple random sampling (SRS)- The English dictionary definition of “random” is usually

something like “without aim or purpose or principle” (Allen, 1985, p. 613).

- In statistics it is NOT this.

SRS = Every sample being equally likely

Example with a small population: Pizza Parlour

Five possible toppings: mushrooms, peppers, olives, sausage, and pepperoni.

They also have a special price for large pizzas with any two toppings of your choice.

Population size N=5 Sample size n=2

Simple Random Sample (SRS)

All samples equally likely (approx. 10% picked)

Mushrooms & peppers Mushrooms & olives

Mushrooms & pepperoni Mushrooms & sausagePeppers & olives Peppers & pepperoniPeppers & sausage Olives & pepperoniOlives & sausage Pepperoni & sausage

Characteristics of SRS

SRS tells us some likely characteristics if we collected lots of samples (frequentist statistical philosophy).

Pr(peppers) = 40%Pr(veggie pizza) = 30%

Statistics tell us how far off we can be from the predicted value before we think that there is a systematic bias (like having a vegetarian in the group)

Are all toppings equally likelya definition of SRS?

It is necessary, but not sufficient: Mushrooms & peppers

Sausage & olives Olives & pepperoni Pepperoni & peppers

Sausage & mushrooms Random sample of this has each topping picked

about 40% of the time, BUT

Pr(veggie) = _________ (YOU fill in the blank)

SRS is Rare (e.g., UK national lottery)

The population is the 49 balls and six are sampled. Any combination of the six balls is equally possible.

There are about 14 million possible combinations

Assuming SRS, the probability of the sample of six balls having certain characteristics can be calculated.

Pr(all six balls even) slightly less than 1%

Biased Sampling (Fienberg, 1971)

• Being drafted for the Vietnam War• Not intended to be an SRS

– “One of equal and uniform treatment for all men in like circumstances” (President Johnson, 6-3-1967)

• Drafted by birthday – 366 capsules

• Filled 31 with Jan. dates, put in box and pushed to one end. Repeated with Feb., pushing them to same end. etc.

• Shook “several times,” carried up and down stairs, poured into a bowl.

• More likely to be drafted if born in later months.

Alternatives to Simple Random Sampling

• Tend to decrease the precision of estimates

• Cluster sampling• Quota sampling• Convenience sampling

Cluster Sampling

Quota Sampling: One meat and one veg

Olives & sausage Mushrooms & pepperoniPeppers & pepperoni Peppers & sausageOlives & pepperoni Mushrooms & sausage

• Popular (cheaper). – Insures at least on some characteristic equal

• Different Characteristics– Pr(sausage)=50%– Pr(mushroom)=33%

• Often produces a bias

Convenience/Opportunity Sample

• Popular in psychology• Difficult to generalize• Used with experiments with random allocations

– Falsifying hypotheses– Local causal inference sometimes possible

Practicalities

• Experimental versus Non-experimental – More care in sampling necessary for quasi-

experimental and non-experimental research.

• Myth: You need to offer money.– It helps, but ...

• Students– Posters, including Departmental Board (give

details)– Cafeterias/Coffee rooms– Dorm Rooms– Existing lists

• Public– Electoral role– Phone books– Airport waiting rooms– On buses and trains– In take-aways– Launderettes

• Recruit people when you think that they have time

Students versus Other

• Students more homogeneous (ie., similar)– power increased for experiments– for non-experiments, want heterogeneity on main

variables• Easier to contact

• but may be too knowledgeable • may talk with each other

Is Sampling just about people?(and pizza toppings)

• Sampling the particular context that is used– Often choose just a particular situation ... will there

be differences among situations• Sampling the particular stimuli (Clark, 1973)

– Will the choice make a difference on inference

Allocation

• The “Gold Standard” for Allocation: Random• Random allocation

– Does not mean each person equally likely to be in each condition.

• Easier than random sampling so usually done.

Festinger & Carlsmith (1959)

Sampling Allocating Measuring Group - $1 -> Rating /Population -> Sample

\ Group - $20 -> Rating

-5 to + 5 scale: $1: -0.05, $20: +1.35

The Lanarkshire Milk Experiment“Student” (1931)

• In 1930 an experiment with 20,000 children in Scotland, to test how giving children milk affected height and weight.

• Children’s height and weight were measured at both the beginning and end of the experiment.

• The teachers were supposed to decide “randomly” who were in the control group, and therefore received no milk, and who were in the experimental group, and received either raw or pasteurised milk.

Was Random allocation used?

Teachers were allowed to substitute well fed or ill nourished children if the control and experimental groups did not appear even in their classrooms.

it would seem probable that the teachers, swayed by the very human feeling that the poorer children needed the milk more than the comparatively well to do, must have unconsciously made too large a substitution of the ill-nourished among the ‘feeders’ and too few among the ‘controls’. (1931, p. 399)

Were the groups the same?

The initial heights and weights of the “control” children taller/heavier.

“though planned on the grand scale, organised in a thoroughly business-like manner and carried through with the devoted assistance of a large team of teachers, nurses and doctors, [it] failed to produce a valid estimate of the advantage of giving milk to children” (“Student”, 1931, p. 406).

Modern Examples of RCT

• RCT means randomized controlled trial• Examples (Berger, 2005)

– File cabinets being broken into– Envelopes being held up to the light– Predicting placement– Berger provides numerous examples of pre-treatment

differences

Summary

• Sampling from a population and allocating people into conditions are important scientific processes.

• Sampling is critical for estimating population values. Simple random sampling is preferred, but deviations often necessary.

• Random allocation should be done.