Producing Data

Chapter 5

Gathering Data

Sample survey Observational study Experiment

Before explaining each of these we must be familiar with some vocabulary.

Vocabulary

Treatments/Explanatory Variables Units Random Assignment

Treatments/Explanatory variables

Explanatory Variables: Attempts to explain the observed outcome. Explains differences in response variable.

Treatment: Any particular combination of values for the explanatory variables; conditions you want to assign or compare.

Units

A unit is a person, animal, plant or thing or groups of people animals, plants or things which is actually being studied by a researcher. The basic objects upon which the study or experiment is carried out.

Units

For example, a person; a monkey; a sample of soil; a pot of seedlings; a cage of rats; a day care center, a child in the day care center, a school, a student at the school, a doctor's practice, a person in the doctors practice.

Random Selection

Randomization is the process by which experimental units are allocated to treatments; that is, by a random process and not by any subjective and hence possibly biased approach. The treatments should be allocated to units in such a way that each treatment is equally likely to be applied to each unit.

Observational study

A type of study in which individuals are observed and certain outcomes are measured. No attempt is made to affect the outcome (for example, no treatment is given).

Observational study

No treatments No random selection of units to treatments No random selection of units from

population of interest

Example: Observational study

On March 8, 1983, Newsweek announced, “A really bad Hair Day: Researchers link baldness and heart attacks” (p62) The article reported that “men with typical male pattern baldness…are anywhere from 30 to 300 percent more likely to suffer a heart attack than men with little or no hair loss at all.”

Example: Observational study

The report was based on an observational study conducted by researchers at Boston University School Medicine. They compared 665 men who had been admitted to the hospital with their first heart attack to 772 men in the same age group (21 to 54 years old) who had been admitted to the same hospitals for other reasons. There were 35 hospitals involved, all in eastern Massachusetts and Rhode Island.

Example: Observational study

The study found that the percentage of men who showed some degree of pattern baldness was substantially higher for those who had had a heart attack (42%) than for those who had not (34%).

Example: Observational study

The authors of the study speculated that a third variable, perhaps a male hormone, might simultaneously increate the risk of heart attacks and the propensity for baldness. With an observational study such as this one, scientists can establish a connection, and they can then look for causal mechanisms in future investigations.

Sample Survey

Units are selected from a population of interest.

Certain outcomes are measured Makes no random assignment of them to

treatments.

An Example: Sample Survey

A farmer wishes to estimate the yield of his 100 acre field. He randomly selects 10 of the acres and observes the number of number of “saleable” produces in each of the 10 acres.

Experiment

Deliberately imposes some treatments on individuals in order to observe their responses.

Assigns all units randomly to treatments or treatments to trials.

May or may not select units at random from the population

How can you differentiate observational studies from sample surveys from

experiments?

Are treatments imposed? Are units randomly assigned to

treatments? Are units randomly selected from

population of interest?

Sample

Survey

Random selection of units from population

Imposed treatment?

Random assignment of units to treatments

Observational

Study Experiment

Yes NO Maybe

Yes

Yes

NO NO

NO NO

Is this an experiment? What is the treatment? Are units randomly assigned to

treatments?

Suppose we wanted to study effects of gender on pulse rate for students in our class. Our population is 30 students in the class. We randomly select 10 of them to participate. We compare average pulse rate of males with pulse rate of females.

It is not an experiment! Sometimes we cannot assign treatment to units!

Suppose we wanted to study effects of gender on pulse rate. Our population is 30 students in a class. We randomly select 10 of them to participate. Can we randomly assign 5 of them to be males and the others to be females? NO! It is not an experiment. The units of study cannot be randomly assigned to treatments! It is a sample survey!

Is this an experiment? Why or why not?What is the treatment?

In 1973 a study was published that compared the purity of intravenous fluids manufactured by three different drug companies. Six different samples were randomly selected from each of the three companies and the number of contaminant particles were measured.

Scope of inference

The scope of inference depends on the manner in which the researchers randomized.

Results can be used how?

Sample Survey

Identify associations among variables for the populations from which the units were randomly selected

Results can be used how?

Observation Study

Identify associations among variables for the units in the study only.

Results can be used how?

Experiment

Allows researchers to draw cause-and –effect conclusions for either the study’s units (if the experimental units were not randomly selected) or the population (if the experimental units were randomly selected).

Allocation of units to treatment groups

At

Random

Not at

Random

Permitted

Inference

At

Random

Experiment with broad scope of inference

Sample Survey

Inference to population

Not at

Random

Experiment with narrow scope of inference

Observational Study

Inference limited to units in study

Permitted Inference

Casual Inference

Association can be made, but no causal inference

Vocabulary Explanatory Variables: Attempts to

explain the observed outcome. Explains differences in response variable.

Treatment: Any particular combination of values for the explanatory variables; conditions you want to assign or compare

Example 1

Does regularly taking aspirin help protect people against heart attacks? The physicians Health Study was a medical experiment that helped answer this question. In fact, the Physicians Health study looked at the effects of two drugs: aspirin and beta carotene. The subjects were 21,996 male physicians.

How many Treatments are there?

Does regularly taking aspirin help protect people against heart attacks? The 21,996 male physicians took either aspirin and beta carotene, aspirin and a placebo, placebo and a beta carotene or two placebos

Factor 2 Beta caroteneYes No

Yes

Aspirin / Beta

Carotene

Aspirin / Placebo

No

Placebo / Beta

Carotene

Placebo / Placebo

There are 4 treatments: What are they?

There are 2 factors: aspirin or beta carotene

There are 2 levels for each factor: Yes or NO

There are three explanatory variables. What are they? Aspirin Beta carotene Placebo

Factorial Design

A factorial design is used to evaluate two or more factors simultaneously. The treatments are combinations of levels of the factors. The advantages of factorial designs over one-factor-at-a-time experiments is that they are more efficient and they allow interactions to be detected.

Example 2

Does the type of lighting or the type of music in the dentists waiting room have an effect on the anxiety level of the patient? Consider that there are 3 types of lighting: Low, med and high and 3 types of music: jazz, pop, and classical.

Example 2

How many treatments are there? How many factors are there? How many levels of each factor are there?

Two factorial design3x3 factorial design

Low Med High

Jazz

Pop

classical

Vocabulary Response Variable: Data collected is the

response variable. Measures outcome of the study

What was the response variable?

In the Physicians health study where the effects of aspirin and beta carotene on heart attack rate was studied, what was the response variable?

Experimental Units

The smallest object to which a treatment will be applied at random.

What are the experimental Units?

A study conducted by researchers exposed one hundred mice at random to cell phone radiation for two half-hour periods each day for eighteen months and fitted another one hundred mice at random with the same type of antennas, which never had the power turned on.

What are the experimental Units?

Suppose that instead of randomizing the individual mice to cell phone exposure or no cell phone exposure, the researchers had placed one hundred mice in one cage and then the other hundred mice in another cage. They randomly selected which cage of mice would get the radiation

What are the experimental units?

A researcher is interested in comparing 3 different gasoline additives with respect to automobile performance as measured by gas mileage. The experimenter uses a single car with an empty tank. One gallon of gas with one of the additives will be poured into the tank and the car driven until it runs out of gas. This is repeated 30 times – 10 times with each additive.

What is the treatment?

A researcher is interested in comparing 3 different gasoline additives with respect to automobile performance as measured by gas mileage. The experimenter uses a single car with an empty tank. One gallon of gas with one of the additives will be poured into the tank and the car driven until it runs out of gas. This is repeated 30 times – 10 times with each additive.

Three Principals of Experimental Design

1. Control

2. Randomization

3. Replication

Control

Control the effects of lurking variables on the response, most simply by comparing two or more treatments.

Randomize

Use impersonal chance to assign experimental units to treatments

Replicate

Replicate each treatment on many units to reduce chance variation in results.

For a given experiment, you should be able to:

Identify the response variable and indicate how they propose to measure it.

Determine the experimental units-the individuals or groups of individuals that they will randomly assign to treatments;

Define the scope of inference-the population to which the experiment’s results can be legitimately applied.

Identify the treatments, factors, levels and explanatory variables.