GATHERING DATA

Chapter 4

4.1 Experiment or Observe?

1. Population versus

Sample

2. Types of Studies:

Experimental and

Observational

3. Comparing

Experimental and

Observational Studies

Learning Objectives:

www.consumersenergy.com

Population and Samples

Population: Subjects of interest

Sample: Subset for whom we have data

Often want answers

about large group but

can’t measure all, so a

subset is chosen

Use statistical techniques

to make conclusions

Observational Study

Merely observe values of

response and

explanatory variables

without doing anything

to the subjects

creativejuice.files.wordpress.com

Sample Survey

Select sample

and interview

Observational

study

Census is survey

of entire

population

www.law.georgetown.edu

Experiment

Imposes certain

conditions and

observes outcomes

Experimental

conditions =

explanatory

variable values =

treatments www.austinchronicle.com

Experiments and Observational Studies

Experiment reduces lurking variables and thus outside influences

Experiments establish cause and effect, unlike observational studies

Some experiments impractical because of ethics, time, money, etc.

www.purbeck.gov.uk

4.2 What are Good and Poor Ways to Sample?

Learning Objectives:

1. Sampling Frame &

Sampling Design

2. Simple Random Sample

3. Random number table

4. Margin of Error

5. Convenience Samples

6. Types of Bias in Sample

Surveys

7. Key Parts of Sample

Survey artstuff.typepad.com

Sampling Frame & Sampling Design

Sampling frame – list of

subjects in (hopefully

total) population

Sampling design

determines how sample

is selected

www.nedarc.org

Simple Random Sampling (SRS)

Random Sampling –

best way to get

representative sample

Simple Random Sample

– each possible sample

of set size n has equal

chance of being

selected

www.youthnoise.com

SRS: Table of Random Numbers

Pg. A6 of text

1. Number subjects

2. Select numbers from random number table or

random number generator

3. Include subjects with random numbers selected

Margin of Error for Population Percentages

Margin of Error –

how well sample

predicts population

With n subjects in

SRS, margin of error

is about

1100%

n

Example: Margin of Error

A survey result

states: “The

margin of error

is plus or minus

3 percentage

points”

2010 British PM Election

i.telegraph.co.uk

Convenience Samples: Poor Ways to Sample

Convenience Sample:

survey sample that’s

easy to get

Unlikely to represent

population

Often severe biases

Results apply only to

observed subjects

www.revophth.com

Convenience Samples: Poor Ways to Sample

Volunteer Sample:

most common

convenience sample

where subjects

volunteer – not

representative

www.stlouiscenterforclinicalresearch.com

Types of Bias in Sample Surveys

Bias: Favoring parts of population

1. Sampling Bias: from sampling method (e.g., nonrandom samples)

2. Nonresponse bias: some subjects cannot be reached or decline

3. Response bias: subject gives incorrect response or question is misleading

4.bp.blogspot.com

Key Parts of a Sample Survey

1. Identify entire

population

2. Construct sampling

frame

3. Random sampling

design to get n subjects

from frame

4. Avoid sampling bias

Cross-stitch Sampler

www.vam.ac.uk

4.3 What are Good and Poor Ways to Experiment?

Learning Objectives:

1. Identify elements of

experiment

2. Experiments

3. 3 components of good

experiments

4. Blinding the study

5. Statistical Significance

6. Generalizing results of

the study

reactorfire.files.wordpress.com

Elements of an Experiment

Experimental units: Subjects

Treatment: Conditions imposed on subjects

Explanatory variable: Defines groups and treatments

Response variable: Outcome

www.clinplus.com

Experiments

Impose treatments on

subjects to observe

responses

Goal: compare effects of

treatments on response

Randomized experiments –

subjects randomly

assigned to treatments

www.privatemdlabs.com

Placebo effect

Placebo – fake

treatment; sugar

pill

Placebo effect –

improving not from

real treatment but

from belief that he

or she should

improve

www.scientificamerican.com

3 Components of a Good Experiment

Control or Comparison Group

Randomization

Replication

pages.cs.wisc.edu

Principle 1: Control or Comparison Group

Helps analyze effectiveness of primary treatment

Placebo removes lurking variables

Control group gets placebo

Clinical trials may compare new treatment with existing

www.redicecreations.com

Experiments should compare treatments rather than effect of single treatment

Example: 400 volunteers asked to quit smoking with some taking and some not taking antidepressant

Control or Comparison Group

www.gpaulmckinneyjr.com

Principle 2: Randomization

1. Eliminates bias from researcher assigning subjects

2. Balances groups on known and lurking variables

colorectal.surgery.ucsf.edu

Principle 3: Replication

1. Reduces difference due to ordinary variation or chance

2. Increases chance that results show true difference

www.printercopierblog.com

Blinding the Experiment

Blind – subjects unaware

of which treatment used

Double-Blind Experiment

- Neither subjects nor

investigators know which

treatment

Controls bias from

respondent and

experimenter

gridskipper.com

Statistically Significant

Difference –

Observed

difference is larger

than expected from

chance

Statistical Significance

Generalizing Results

Goal of

Experimentation –

Analyze association

between treatment

and response for

entire population

Generalize only to

population

represented by study

static.howstuffworks.com

4.4 Other Ways to Conduct Experimental and

Observational Studies

Learning Objectives

Understand basic principles

of the following:

1. Sample Surveys

2. Two Types of

Observational Studies

3. Multifactor Experiment

4. Matched Pairs Design

5. Randomized Block

Design

www.greatstuffforkids.com

Sample Surveys: Random Sampling Designs

Alternative to

experiments

1. Simple Random

Sampling

2. Cluster Random

Sampling

3. Stratified

Random

Sampling codetechnology.files.wordpress.com

Cluster Random Sample

1. Divide population

into large

number of

clusters, such as

city blocks

2. Select simple

random sample

of clusters

3. Use all subjects in

clusters as sample

Advantages

Sampling frame unavailable

Cost

Disadvantage

Need larger sample size for same reliability

Cluster Random Sample

reactorfire.files.wordpress.com

Stratified Random Sample

1. Divide the

population into

groups, strata

2. Select SRS from

each strata

3. Combine samples

from each for total

sample

www.nedarc.org

Advantage

Ensures stratum

representation

Disadvantage

Need sampling

frame and to

which stratum

each subject

belongs

Stratified Random Sample

www.smh.com.au

Comparing Random Sampling Methods

Types of Observational Studies

1. Sample Survey:

current

2. Retrospective

Study: past

3. Prospective Study:

future

Cause not proven,

but studies can

support beliefs

Back to the Future

youtube.com/watch?v=c8Rngy90Q14

Retrospective Case-Control Study

www.lisisoft.com

Studying sunlight

exposure and multiple

sclerosis connection…

Retrospective

Cases – have MS

Controls or don’t

Explanatory variable –

low sun or not

Prospective Case-Control Study

www.dubaiinternetmarketing.com

Studying

effects of

vegetarian

diet on heart

disease…

Prospective

Cases – have heart disease

Controls or don’t

Explanatory variable –

vegetarian or not

Multifactor Experiments

Single experiment

analyzes two or more

factors

Learn more since

combinations may

affect response

biobreak.files.wordpress.com

Matched Pairs Design

Subjects are somehow matched

Husband/wife, two plots in same field, etc.

Same individual – crossover design

Randomly assign or randomize order of treatments

Reduces effects of lurking variables

www.marshallhammondwedding.info

Randomized Block Design

Block – subjects with common characteristics

Randomized Block Design, RBD – within each block,

randomly assign to treatments

Image Sources

Statistics: The Art and Science of Learning from Data, 2nd Edition, Agresti

and Franklin

http://www.consumersenergy.com/uploadedImages/CEWEB/YOUR_BUSI

NESS/lab-tech-with-beakers.jpg

http://creativejuice.files.wordpress.com/2007/10/voyeur.jpg

http://www.law.georgetown.edu/ist/ISTalk/2009/pablo.jpg

http://www.austinchronicle.com/binary/c506/hypo.needle.jpg

http://www.privatemdlabs.com/newsimages/Medicine_2248_19541486

_0_0_7037531_300.jpg

http://www.lisisoft.com/imglisi/5/Screensavers/56804sunset.jpg

http://artstuff.typepad.com/.a/6a00e54ff1492b88340120a59ac8fb9

70b-800wi

http://rochesterchiro.files.wordpress.com/2008/06/neworleans.jpg

http://www.dubaiinternetmarketing.com/wp-

content/uploads/2009/12/Vegetarian-Restaurants-Dubai.png

http://reactorfire.files.wordpress.com/2009/11/20090617155350_stan

ford2.jpg

http://www.nedarc.org/nedarc/collectingData/participantSelection/pro

babilitySampling/images/stratifiedSample.gif

http://www.youtube.com/watch?v=c8Rngy90Q14

http://www.psdgraphics.com/wp-content/uploads/2009/05/bulls-

eye.jpg

http://biobreak.files.wordpress.com/2009/04/wondertwins.jpg

http://www.marshallhammondwedding.info/contentimage/bride-and-

groom-portrait.jpg

http://s3.amazonaws.com/picable/2009/05/25/981127_Drug-abuse-

by-teens_620.jpg

http://www.nedarc.org/nedarc/collectingData/participantSelection/ima

ges/subsetPopulation.gif

http://www.purbeck.gov.uk/images/bird%20watching.jpg

http://www.youthnoise.com/Assets/Image/Gill%27s%20images/coin_fli

p.jpg

http://hoppingintopuddles.files.wordpress.com/2008/03/random-

numbers.jpg

http://i.telegraph.co.uk/telegraph/multimedia/archive/01630/electionD

ay_1630464c.jpg

http://www.revophth.com/publish/images/1_14244_0.gif

http://www.stlouiscenterforclinicalresearch.com/slccrtest_files/volunteer_g

roup.jpg

http://4.bp.blogspot.com/_BSwhxCnDX-

s/S7roA8X0fKI/AAAAAAAAAvE/MQFz6vSJ9pg/s1600/BiasLogo

White.jpg

http://www.smh.com.au/ffximage/2006/09/22/sydneystudents_widewe

b__470x317,0.jpg

http://codetechnology.files.wordpress.com/2008/06/survey-form.jpg

http://www.greatstuffforkids.com/files/2235105/uploaded/76020-

ABCBlocks.jpg

http://static.howstuffworks.com/gif/population-six-billion-1.jpg

http://fajr.files.wordpress.com/2009/01/scales.jpeg

http://gridskipper.com/assets/resources/2006/11/blindfolded.jpg

http://www.printercopierblog.com/wp-

content/uploads/2008/09/office-copier.jpg

http://colorectal.surgery.ucsf.edu/media/1160745/randomization.gif

http://www.scientificamerican.com/media/inline/placebo-effect-a-cure-

in-the-mind_1.jpg

http://www.redicecreations.com/ul_img/6999placebo_big.jpg

http://www.gpaulmckinneyjr.com/Files/images/quit_smoking.gif

http://www.clinplus.com/Portals/53286/images/iStock_000005903700

XSmall-resized-600.jpg