Causes of Error in Sampling

Sampling Error

• Sampling error is error caused by the way you chose your sample– Volunteer Sampling & Convenience Sampling– Causes Bias

• How do you reduce?– Use randomness in choosing

your sample

Random Sampling Error• Random Sampling Error– You sample may randomly have a higher or lower

percentage of females, college educated people, Hispanics, etc. than what what is found in your population.

– Causes Variation• How do you reduce?– Choose a larger sample sample size.

Random Sampling Error• If your sample ≤ 10% of the population than

your error is about n =

sample size

– True no matter how big the population is• If you sampled 10,000 Indiana residents,

population 6.5 million, what is your error?

• If you sampled 10,000 South Bend residents, population 101,658 what is your error?

Random Sampling Error• If your sample ≤ 10% of the population than

your error is about n =

sample size

• If your sample ≥ 10%, then your error is better than this equation, but we won’t get into that until chapter 9.

Confidence Interval• If you calculate your error to be 1%, your

actual error could be larger. That 1% is what we call a confidence interval. You are 95% confident that the error between your measured result and the actual result is 1%.

• If the results from your study show that the percent of Americans that believe in Heaven is 88 + 5%, what that means is you are 95% confident that between 83% and 93% of the population believes in Heaven.

Undercoverage• Undercoverage is when some groups in the

population are left out of the process of choosing the sample.

• What groups CAN’T be contacted by a survey where you call random people and ask their opinion?– Amish– Homeless– People serving overseas– Prison Inmates– Unlisted numbers & other people without phones

Undercoverage• Undercoverage is when some groups in the

population are left out of the process of choosing the sample.

• How do you reduce/prevent undercoverage– Census (poll everyone)– Reduce bias in your sampling (avoid convenience

& volunteer sampling)– Stratified Sample: Split the population in groups

and then sample each group.• Example: Instead of randomly selection 100 people,

randomly select 50 men and 50 women

Nonsampling Error• Processing Error– Enter a number wrong in Excel– Math Error

• How do you reduce/prevent– Double Check your Work– Don’t Rush

Nonsampling Error• Response Error – given an incorrect response• Why would someone do that?– Lie• How much do you weigh?• Have you ever used drugs?

– Remember Incorrectly• How many minutes have you watched TV this week?• Where exactly were you at 3:30 PM last Saturday?

– Vague or Confusing Question• How many windows do you have? Do door windows

count?

Response Error• Lying causes bias• How can we reduce lying?– Confidential: the interviewer promises their name

won’t be released with the results– Anonymous: even the interviewer doesn’t know

which response sheet corresponds to which person

– Study them without them knowing, as with the handwashing homework problem• Is this ethical?

Response Error• How can we reduce memory errors?

Response Error• Confusing questions cause bias if they favor

one answer over another. They cause variability if they are just confusing in general, but don’t favor a particular outcome

• How can we reduce people from misunderstanding the question?– Clear, careful and extremely specific wording

Response Error: Question Wording• “Owning stock” means different things to a

Wall Street Banker & a Texan Rancher• 13 % of Americans think we are spending too

much on “assistance to the poor,” but 44% think we are spending too much on “welfare”

• A poll in Scotland showed that 51% would vote in favor for “independence for Scotland,” but 34% would vote in favor for “an independent Scotland separate from the United Kingdom.”

• Assistance & Independence are positive words while Welfare & Separate are negative words

Wording Questions• Loaded questions cause bias– Do you favor banning private ownership of

handguns in order to reduce the rate of violent crime?

– George Bush: great president or greatest president?

– Do you support our president?– Do you agree with all of Obama’s policies?– Do you approve or disapprove of the way Barack

Obama is handling his job as president?

Wording Questions

• Open vs. Closed Questions– Rate Obama’s performance on a scale of 1 to 10

vs. what do you think about Obama’s job as president?

– Limiting people’s options can cause bias– Limiting people’s options can reduce people

misunderstanding the question– Closed Questions are easier to analyze

Non Response

• They refuse to answer your survey. They hang up on you, slam the door in your face or just politely say no.

Nonadherer

• A nonadherer is a generic term for someone who doesn’t follow directions for whatever reason.– Example: They forgot to take the experimental pill

every 6 hours.– Example: They lied on their survey form

• How would you reduce?

Dropouts

• Sometimes your study the same person over several days, weeks, months or even years. Dropouts are people who start doing the experiment, but then stop half way through.