Causes of Error in Sampling

Sampling Error• Sampling error is error caused by the way you

chose your sample– Volunteer Sampling & Convenience Sampling– Causes Bias

• How do you reduce?– Use randomness in choosing

your sample

Sampling ErrorRandom Sampling Error

• Random Sampling Error– You sample may randomly have a higher or lower

percentage of females, college educated people, Hispanics, etc. than what what is found in your population.

– Causes Variation• How do you reduce?– Choose a larger sample size.

Sampling ErrorRandom Sampling Error

• From the bell curve, 95% of the data lie within + ______ standard deviations

• This means that we are 95% confident that the true value is within 2 standard deviations of our measured value.

Std. Dev. for Categorical Variables

2

Sampling ErrorRandom Sampling Error

• If you change the 2 you can find other confidence intervals.– Example: Using 3 yields 99.7% confidence interval– We’ll mostly stick to 95% confidence

• This only finds the expected error from randomness… variability.

• Does not account for bias.

Sampling ErrorUndercoverage

• Undercoverage is when some groups in the population are left out of the process of choosing the sample.

• What groups CAN’T be contacted by a survey where you call random people and ask their opinion?– Amish– Homeless– People serving overseas– Prison Inmates– Unlisted numbers & other people without phones

Sampling Error:Undercoverage

• Undercoverage causes… • How do you reduce/prevent undercoverage– Census (poll everyone)– Reduce bias in your sampling (avoid convenience

& volunteer sampling)– Stratified Sample: Split the population in groups

and then sample each group.• Example: Instead of randomly selection 100 people,

randomly select 50 men and 50 women

bias

Nonsampling Error:Processing Error

• Examples– Enter a number wrong in Excel– Math Error

• How do you reduce/prevent– Double Check your Work– Don’t Rush

• Causes Variation

Nonsampling Error:Response Error

• Response Error – given an incorrect response• Why would someone do that?– Lie• How much do you weigh?• Have you ever used drugs?

– Remember Incorrectly• How many minutes have you watched TV this week?• Where exactly were you at 3:30 PM last Saturday?

– Vague or Confusing Question• How many windows do you have? Do door windows

count?

Nonsampling Error:Response Error

• Lying causes bias• How can we reduce lying?– Confidential: the interviewer promises their name

won’t be released with the results– Anonymous: even the interviewer doesn’t know

which response sheet corresponds to which person

– Study them without them knowing, as with the handwashing homework problem• Is this ethical?

Nonsampling Error:Response Error

• How can we reduce memory errors?

Nonsampling Error:Response Error

• Confusing questions cause bias if they favor one answer over another. They cause variability if they are just confusing in general, but don’t favor a particular outcome

• How can we reduce people from misunderstanding the question?– Clear, careful and extremely specific wording

Nonsampling Error:Response Error: Question Wording

• 13 % of Americans think we are spending too much on “assistance to the poor,” but 44% think we are spending too much on “welfare”

• A poll in Scotland showed that 51% would vote in favor for “independence for Scotland,” but 34% would vote in favor for “an independent Scotland separate from the United Kingdom.”

• Assistance & Independence are positive words while Welfare & Separate are negative words

Wording Questions• Loaded questions cause bias– Do you favor banning private ownership of handguns

in order to reduce the rate of violent crime?– George Bush: great president or greatest president?

– Do you support our president?– Do you agree with all of Obama’s policies?– Do you approve or disapprove of the way Barack

Obama is handling his job as president?

Wording Questions: Vague• How many windows are in your house is a

simple enough question…– Does this room have 2 or 12 windows on the wall?– Do the windows on the door count?

• If you’re not specific, everyone is going to interpret the question differently adding a lurking variable and variability to your data.

Wording Questions

• Open vs. Closed Questions– Rate Obama’s performance on a scale of 1 to 10

vs. what do you think about Obama’s job as president?

– Limiting people’s options can cause bias– Limiting people’s options can reduce people

misunderstanding the question– Closed Questions are easier to analyze

Non Response

• They refuse to answer your survey. They hang up on you, slam the door in your face or just politely say no.

Nonadherer

• A nonadherer is a generic term for someone who doesn’t follow directions for whatever reason.– Example: They forgot to take the experimental pill

every 6 hours.– Example: They lied on their survey form

• How would you reduce?

Dropouts

• Sometimes your study the same group over several days, weeks, months or even years. Dropouts are people who start doing the experiment, but then stop before it is complete.