Task 4: Identifying and preventing bias

OCR Nationals Level 3 Unit 3

March 2012 M Morison

Objectives

Know what is meant by ‘bias’ in a study

Understand that bias needs to be eliminated from a study so that the results are accurate, valid and reliable

Be able to identify areas of potential bias in your study, and explain the steps that you will take to minimise this.

March 2012 M Morison

Deadline for this unit:

 A02 - Design and Carry Out a data collection activity

Monday April 2nd

March 2012 M Morison

Pass/Merit/Distinction

Pass Candidates will identify some areas of possible or potential bias in their study.

Merit Candidates will identify some areas of potential bias in their study, and list some steps that could be taken to minimise this.

DistinctionCandidates will explain the steps they have taken to eliminate bias from their study.

4

March 2012 M Morison

What is Bias?

When collecting data, it is important to avoid bias.

Definition: Bias is an unintended influence on a data-gathering method.

There are several types of bias that may occur if you are not careful when gathering your data.

Here are a few types of bias that may happen if you aren't careful.

March 2012 M Morison

Sampling Bias

When, for whatever reason, the chosen sample does not accurately represent the population.

Sometimes, even when we use random sampling, we may encounter a sampling bias.

To reduce this risk, we should use an appropriate sampling technique with a large enough sample.

March 2012 M Morison

Bias Due to Unrepresentative Samples

Undercoverage occurs when some members of the population are inadequately represented in the sample.

A classic example of undercoverage is the Literary Digest voter survey, which predicted that Alfred Landon would beat Franklin Roosevelt in the 1936 presidential election. The survey sample suffered from undercoverage of low-income voters, who tended to be Democrats.How did this happen? The survey relied on a convenience sample, drawn from telephone directories and car registration lists. In 1936, people who owned cars and telephones tended to be more affluent. Undercoverage is often a problem with convenience samples.

March 2012 M Morison

Non-Response Bias

When some surveys are not returned, thus influencing the results of the survey.

To avoid this, find a creative way to make sure everyone returns their survey.

March 2012 M Morison

Household Bias

When one type of respondent is overrepresented because groupings of different sizes are polled equally. For example, if we want to know the favourite television show of all people at Meden and we surveyed 20 teachers and 20 students, we would be over-representing the teachers.

March 2012 M Morison

Household Bias (cont’d)

Since there are many more students than teachers, we should survey many more students. (We should survey a certain PERCENTAGE of each group being surveyed to ensure that each group is represented equally.)

March 2012 M Morison

Response Bias

When the sampling method itself influences the results.

For example, if a survey had leading questions or the person conducting the survey tried to influence the survey takers.

March 2012 M Morison

Techniques for Preventing Response Bias

Write questions that are clear, precise, and relatively short

Because every question is measuring something, it is important for each to be clear and precise.

Your goal is for each respondent to interpret the meaning of each survey question in exactly the same way.

If your respondents are not clear on what is being asked in a question, their responses may result in data that cannot or should not be applied to your survey goals.

Keep questions short; long questions can be confusing and stressful for respondents.

March 2012 M Morison

Do not use “loaded” questions A loaded or leading

question biases the response given by the participant.

A loaded question is one that contains loaded words. For example, politicians often avoid the loaded word “environmentalist” because it creates a negative reaction in some people regardless of the content of the statement.

March 2012 M Morison

Do not use “leading” questionsA leading question is phrased in such

a way that suggests to the respondent that the researcher expects a certain answer:

Example Don’t you agree that social workers should earn more money than they currently earn? Yes, they should earn more No, they should not earn more Don’t know/no opinion

March 2012 M Morison

Avoid double-barrelled questions

A double-barrelled question combines two or more issues or attitudinal objects in a single question.

Example

Do you think professors should have more contact with university staff and university administrators?

March 2012 M Morison

Double-barrelled questions (cont’d)

Clearly, this question asks about two different issues:

1. Do you think professors should have more contact with university staff? AND

2. Do you think that teachers should have more contact with university administrators?

March 2012 M Morison

Double-barrelled questions (cont’d)

Combining the two questions into one question makes it unclear which attitude is being measured, as each question may elicit a different attitude.

Tip: If the word “and” appears in a question, check to verify whether it is a double-barrelled question.

March 2012 M Morison

Avoid double negatives

Example Do you agree or disagree with the following

statement?Teachers should not be required to supervise their students during recess.

If the respondent disagrees, you are saying you do not think teachers should not supervise students. In other words, you believe that teachers should supervise students. If you do use a negative word like “not”, consider highlighting the word by underlining or bolding it to catch the respondent’s attention.

March 2012 M Morison

Construction Principles

Use mutually exclusive and exhaustive response categories for closed-ended questionsCategories are mutually exclusive when there is no overlap: Example

What is your current age? 10 or less 10 to 20 20 to 30 30 to 40 40 to 50 50 or greater

March 2012 M Morison

A better example:

What is your current age? (Check one box only.)

Less than 18 18 to 29 30 to 39 40 to 49 50 or older