agresti/franklin statistics, 1 of 56 chapter 4 gathering data learn …. how to gather “good”...

Agresti/Franklin Statistics, 1 of 56

Chapter 4Gathering data

Learn ….

How to gather “good” data

About Experiments and Observational Studies


Section 4.1

Should We Experiment or Should we Merely Observe?


Population, Sample and Variables

Population: all the subjects of interest

Sample: subset of the population -data is collected on the sample

Response variable: measures the outcome of interest

Explanatory variable: the variable that explains the response variable


Types of Studies

Experiments

Observational Studies


Experiment

A researcher conducts an experiment by assigning subjects to certain experimental conditions and then observing outcomes on the response variable

The experimental conditions, which correspond to assigned values of the explanatory variable, are called treatments


Observational Study

In an observational study, the researcher observes values of the response variable and explanatory variables for the sampled subjects, without anything being done to the subjects (such as imposing a treatment)


Example: Does Drug Testing Reduce Students’ Drug Use?

Headline: “Student Drug Testing Not Effective in Reducing Drug Use”

Facts about the study:

• 76,000 students nationwide

• Schools selected for the study included schools that tested for drugs and schools that did not test for drugs

• Each student filled out a questionnaire asking about his/her drug use



Conclusion: Drug use was similar in schools that tested for drugs and schools that did not test for drugs



What were the response and explanatory variables?



Was this an observational study or an experiment?


Advantages of Experiments over Observational Studies

We can study the effect of an explanatory variable on a response variable more accurately with an experiment than with an observational study

An experiment reduces the potential for lurking variables to affect the result


Experiments vs Observational Studies

When the goal of a study is to establish cause and effect, an experiment is needed

There are many situations (time constraints, ethical issues,..) in which an experiment is not practical


Good Practices for Using Data

Beware of anecdotal data

Rely on data collected in reputable research studies


Example of a Dataset

General Social Survey (GSS): • Observational Data Base

• Tracks opinions and behaviors of the American public

• A good example of a sample survey

• Gathers information by interviewing a sample of subjects from the U.S. adult population

• Provides a snapshot of the population


Section 4.2

What Are Good Ways and Poor Ways to Sample?


Setting Up a Sample Survey

Step 1: Identify the Population

Step 2: Compile a list of subjects in the population from which the sample will be taken. This is called the sampling frame.

Step 3: Specify a method for selecting subjects from the sampling frame. This is called the sampling design.


Random Sampling

Best way of obtaining a representative sample

The sampling frame should give each subject an equal chance of being selected to be in the sample


Simple Random Sampling

A simple random sample of ‘n’ subjects from a population is one in which each possible sample of that size has the same chance of being selected


Example: Sampling Club Officers for a New Orleans Trip

The five offices: President, Vice-President, Secretary, Treasurer and Activity Coordinator

The possible samples are:

(P,V) (P,S) (P,T) (P,A) (V,S)

(V,T) (V,A) (S,T) (S,A) (T,A)


The possible samples are: (P,V) (P,S) (P,T) (P,A) (V,S) (V,T) (V,A) (S,T) (S,A) (T,A)

What are the chances the President and Activity Coordinator are selected?

a. 1 in 5

b. 1 in 10

c. 1 in 2


Selecting a Simple Random Sample

Use a Random Number Table

Use a Random Number Generator


Methods of Collecting Data in Sample Surveys

Personal Interview

Telephone Interview

Self-administered Questionnaire


How Accurate Are Results from Surveys with Random Sampling?

Sample surveys are commonly used to estimate population percentages

These estimates include a margin of error


Example: Margin of Error

A survey result states: “The margin of error is plus or minus 3 percentage points”

This means: “It is very likely that the reported sample percentage is no more than 3% lower or 3% higher than the population percentage”

Margin of error is approximately:1

100%n


Be Wary of Sources of Potential Bias in Sample Surveys

A variety of problems can cause responses from a sample to tend to favor some parts of the population over others


Types of Bias in Sample Surveys

Sampling Bias: occurs from using nonrandom samples or having undercoverage

Nonresponse bias: occurs when some sampled subjects cannot be reached or refuse to participate or fail to answer some questions

Response bias: occurs when the subject gives an incorrect response or the question is misleading


Poor Ways to Sample

Convenience Sample: a sample that is easy to obtain

• Unlikely to be representative of the population

• Severe biases my result due to time and location of the interview and judgment of the interviewer about whom to interview


Poor Ways to Sample

Volunteer Sample: most common form of convenience sample• Subjects volunteer for the sample

• Volunteers are not representative of the entire population


A Large Sample Does Not Guarantee An Unbiased Sample

Warning:


Section 4.3

What Are Good Ways and Poor Ways to Experiment?


An Experiment

Assign each subject (called an experimental unit ) to an experimental condition, called a treatment

Observe the outcome on the response variable

Investigate the association – how the treatment affects the response


Elements of a Good Experiment

Primary treatment of interest

Secondary treatment for comparison

Comparing the primary treatment results to the secondary treatment results help to analyze the effectiveness of the primary treatment


Control Group

Subjects assigned to the secondary treatment are called the control group

The secondary treatment could be a placebo or it could be an actual treatment


Randomization in an Experiment

It is important to randomly assign subjects to the primary treatment and to the secondary (control) treatment

Goals of randomization: • Prevent bias

• Balance the groups on variables that you know affect the response

• Balance the groups on lurking variables that may be unknown to you


Blinding the Study

Subjects should not know which group they have been assigned to – the primary treatment group or the control group

Data collectors and experimenters should also be blind to treatment information


Example: A Study to Assess Antidepressants for Quitting Smoking

Design:

• 429 men and women

• Subjects had smoked 15 cigarettes or more per day for the previous year

• Subjects were highly motivated to quit



Subjects were randomly assigned to one of two groups:• One group took an antidepressant daily

• Second group did not take the antidepressant (this group is called the placebo group)



The study ran for one year

At the end of the year, the study observed whether each subject had successfully abstained from smoking or had relapsed



Results after 1 year:

• Treatment Group: 55.1% were not smoking

• Placebo Group: 42.3% were not smoking

Results after 18 months:• Antidepressant Group: 47.7% not smoking

• Placebo Group: 37.7% not smoking

Results after 2 years:• Antidepressant Group: 41.6% not smoking

• Placebo Group: 40% not smoking



Question to Think About: Are the differences between the two groups statistically significant or are these differences due to ordinary variation?


Section 4.4

What Are Other Ways to Conduct Experimental and Observational

Studies?


Multifactor Experiments

Multifactor Experiments: have more than one categorical explanatory variable (called a factor).


Example: Do Antidepressants and/or Nicotine Patches Help Smokers Quit?


Matched-Pairs Design

Each subject serves as a block

Both treatments are observed for each subject


Example: A Study to Compare an Oral Drug with a Placebo for Treating Migraine Headaches

Subject Drug Placebo

1 Relief No Relief First matched pair

2 Relief Relief

3 No Relief No Relief


Blocks and Block Designs

Block: collection of experimental units that have the same (or similar) values on a key variable

Block Design: identifies blocks before the start of the experiment and assigns subjects to treatments with in those blocks


Experiments vs Observational Studies

An Experiment can measure cause and effect

An observational study can yield useful information when an experiment is not practical

An observational study is a practical way of answering questions that do not involve trying to establish causality



A well-designed and informative observational study can give the researcher very useful data.

Sample surveys that select subjects randomly are good examples of observational studies.


Random Sampling Schemes

Simple Random Sample: every possible sample has the same chance of selection



Cluster Random Sample: • Divide the population into a large number

of clusters

• Select a sample random sample of the clusters

• Use the subjects in those clusters as the sample



Stratified Random Sample:• Divide the population into separate groups,

called strata

• Select a simple random sample from each strata



Well-designed observational studies use random sampling schemes


Retrospective and Prospective Studies

Retrospective study: looks into the past

Prospective study: follows its subjects into the future


Case-Control Study

A case-control study is an observational study in which subjects who have a response outcome of interest (the cases) and subjects who have the other response outcome (the controls) are compared on an explanatory variable


Example: Case-Control Study

Response outcome of interest: Lung cancer

• The cases have lung cancer

• The controls did not have lung cancer

The two groups were compared on the explanatory variable:• Whether the subject had been a smoker

agresti/franklin statistics, 1 of 56 chapter 4 gathering data learn …. how to gather “good”...

Documents

agrestifranklin statistics

response variable slide

hisher drug use slide

students drug use

observational study

result slide

treatments slide

treatment slide