OBSERVATIONAL STUDY VS. EXPERIMENT

OBSERVATIONAL STUDY

The researcher observes individuals and measures variables of interest without influencing the responses.

GOAL: to draw conclusions about the corresponding population or about differences between two or more populations

Observational studies of the effect of one variable on another often fail because of confounding between the explanatory variable and one or more lurking variables.

Definition:

A lurking variable is a variable that is not among the explanatory or response variables in a study but that may influence the response variable.

Confounding occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other.

Well-designed experiments take steps to avoid confounding.

EXAMLE

Observe women who take hormones vs. those not taking hormones and note whether a heart attack has occurred.

Lurking variables: Economic status Education Frequency of visiting the doctor

EXPERIMENT

The researcher deliberately imposes some treatment on individuals to measure their responses.

GOAL: to determine whether the treatment CAUSES a change in the response

Can provide evidence for a cause-and-effect relationship

(an observational study cannot draw such

conclusions because we can’t rule out confounding variables.)

WELL-DESIGNED EXPERIMENT

CAUSE AND EFFECTChildren who drink more milk have bigger feet than children who drink less milk. There are three possible explanations for this association:

?

Drinking milk causes children’s feet to be bigger. Having bigger feet causes

children to drink more milk.

A lurking variable is responsible for both.

Lies in the background that may or may not be apparent at the outset but once identified could explain the pattern between the variables.

LURKING VARIABLES

The Language of ExperimentsAn experiment is a statistical study in which we actually do

something (a treatment) to people, animals, or objects (the experimental units) to observe the response. Here is the basic vocabulary of experiments.

Treatment : a specific condition applied to the individuals in an experiment •If an experiment has several explanatory variables, a treatment is a combination of specific values of these variables.

Experimental units: the collection of individuals to which treatments are applied.

•When the units are human beings, they often are called subjects. Sometimes, the explanatory variables in an experiment are called factors. Many experiments study the joint effects of several factors. In such an experiment, each treatment is formed by combining a specific value (often called a level) of each of the factors.

There are now many special courses that claim to prepare you for the SAT. Suppose that you want to evaluate a particular course, using SAT scores to measure the effect of the course. You might find a reasonably large high school where students are offered the chance to take the course, and then compare the SAT scores of those who completed the course with the scores of those who chose not to take it.

DISCUSSION

Suppose that you find that the average SAT score for students who took the course is 30 points higher than for students who didn’t.

Identify each of the elements in this study: population, response variable, treatments

POP- students who took the SATResponse Variable- SAT scoreTreatments- taking the course/ not taking the course

Is this study a true experiment?NO- the students are not randomly assigned to the two treatments

Do you conclude that the course causes an increase in SAT scores? Why or why not?

NO- those who took the course may be unrepresentative due to selection bias. Those who take the course may tend to score higher anyway.

1. A: having hip surgery B: dying within the next 10 yearsLurking variable- person’s age

2. A: the amount of milk a person drinks B: the strength of a person’s bones

3. A: the amount of money a person earns B: the number of years a person went to school

4. A. the number of classes taken with Mrs. Sapp  B. level of happiness

EACH PAIR OF VARIABLES SHOWN ARE STRONGLY ASSOCIATED. DOES A CAUSE B OR DOES B CAUSE A, OR IS THERE A LURKING VARIABLE?

In 1912, Dr. Charles Mayo published an article recommending removal of the thymus as treatment for respiratory problems in children. He made this recommendation even though 1/3 of the children died after the operation. Look at the study:

THYMUS EXAMPLE

CHILD ADULT

LARGE Problems No evidence

SMALL No evidence No problems

AGE

Thymus size

Why is it impossible to know whether a large thymus causes respiratory problems? Age and size of thymus were confounded

Suppose that the surgeons had examined infants without respiratory problems and found that their thymus was generally small.

Have they now proved that a large thymus causes respiratory problems in children?

Design an experiment to determine if removal of the thymus helps children with respiratory problems.

Randomly divide the children with respiratory problems into two groups: one that gets surgery and one that does not. Otherwise treat them exactly alike. Record how many survived under each treatment.

THYMUS STUDY

Pg. 44Why was this a poorly designed experiment?What were some possible extraneous factors?

CALCULUS EXAMPLE

Experimental Units Treatment

Measure Response

NOTE: In the lab environment, simple designs often work well.

Field experiments and experiments with animals or people deal with more variable conditions. Outside the lab, badly designed experiments often yield worthless results because of confounding.

Model for experiments

EXAMPLES

Suppose Starbucks wishes to find out whether the population of MHS students prefer hot or cold, frozen coffee drinks. A random sample of students is selected, and each one is asked to try first hot coffee and then frozen coffee, or vice versa (with the order determined at random). They then indicate which type they prefer.

Experiment or Observational Study?

A researcher is interested in the effects of excessive homework on family dinner nights. She surveys students on their homework load, and the number of family nights they have been required to forego. She concludes that homework load does not directly affect family nights.

Experiment or Observational Study?

Suppose an experiment is designed to investigate the effects of repeated exposure to TV ads. All subjects viewed a 40 minute TV program that included ads for an iPad. Some subjects saw a 30- second commercial; others, a 90-second version. The same commercial was shown either 1, 3, or 5 times during the program. After viewing, all the subjects answered questions about their recall of the ad, their attitude toward the iPad and their intention to purchase it.

Identify the explanatory and response variables:

List all the treatments: