COGS 14B: INTRODUCTION TO STATISTICAL ANALYSIS

TA:

Sai ChowdaryGullapally

scgullap@eng.ucsd.edu

Office Hours:Thursday (Mandeville)3:30PM - 4:30PM(or by appointment)

Feel free to mail anytime regarding doubts.Slides: I am using the amazing slides made by Joey :)

for most of the questions.

Hypothesis Testing:

What are we actually doing?

What is the intuition?

A not so different scenario:Let us assume we have a variable “x” (could be anything like length, heights etc) Further assume that we know “x” has one of the following probability densities given below, but unfortunately we do not know which one is the correct one. So what is the best thing we can do to guess?

TAKE A SAMPLE AND SEE !!!!

And then??

Type I Error:

Type II Error:

H0 is true, but we reject it

(probability set by α)H

1 is true, but we retain H

0(probability given by β!)

Type II error

Sampling Distributions

αβ

hypothesizedTrue

Type I Error:

Type II Error:

α

H0 is true, but we reject it

(probability given by α)H

1 is true, but we retain H

0(probability given by β – more later!)

Power: Probability of detecting a particular effect (i.e., rejecting H0 when it is false).

Type II error

Alternative Hypothesis

H1

POWER

Sampling Distributions

β

hypothesized True

Name some factors that affect the power of a statistical test. Which of these factors does the

experimenter have control over?

Hypothesis Testing Summary (so far)Z test t test

(one sample)t test

(2 independent samples)

Use when:

Statistical hypotheses (two-tailed)

Test statistic

Distribution

You want to compare a sample mean to the mean of a population with a known standard deviation

You want to compare one sample mean to the (hypothesized) mean of a population with an unknown standard deviation

You want to compare means of two independent samples taken from different populations (with unknown standard deviations)

H0: μ = μ

0H

0: μ

1 – μ

2 = 0

z = t =

H1: μ ≠ μ

0H

1: μ

1 – μ

2 ≠ 0

z distribution (standard normal)

t distribution with n-1 degrees of freedom

t distribution with n

1 + n

2 – 2 degrees of

freedom

H0: μ = μ

0

H1: μ ≠ μ

0

A random sample of college seniors received special training on how to take the GRE. After analyzing their scores, an investigator reported a dramatic gain relative to the national average of 500, as indicated by a 95% confidence interval of 507 to 527. Are the following interpretations true or false?

(a) About 95% of all subjects scored between 507 and 527.

(b) The interval from 507 to 527 refers to a set of possible values of the population mean for all students who undergo special training.

(c) The true population mean is definitely between 507 and 527.

(d) This particular interval contains the population mean about 95% of the time.

(e) In practice, we never really know whether the interval from 507 to 527 is true or false.

(f) We can be reasonably confident that the population mean is between 507 and 527.

False

True

False

False

True

True

T-test (repeated measures)Use when:

Statistical hypotheses(two tailed)

Test statistic

Distribution

Hypothesis Testing Summary (so far)

You want to compare means between two groups in a repeated measures design (or paired design)

H0: μ

D = 0

H1: μ

D ≠ 0

t distribution with n – 1 degrees of freedom (n is # of pairs)

The following the are the test performances of the same group of 6 people, obtained while using caffeine and without it. Does Caffeine increase test performance?

The following the are the test performances of the same group of 6 people, obtained while using caffeine and without it. Does Caffeine increase test performance?

H0: μ

D = 0

H1: μ

D > 0

Degrees of Freedom: 6-1=5

tcritical

=2.015t=4.110

Reject H0

ANOVA (one-way, between subjects)

Use when:

Statistical hypotheses

Test statistic

Distribution

Hypothesis Testing Summary (so far)

You want to test whether there are any differences between multiple (>2) population means

H0: μ

0 = μ

1 = μ

2 = …= μ

n

H1: H

0 is false

F distribution (dfbetween

and df

within)

Can you use ANOVA to test a directional hypothesis?

Why not multiple T-tests?

What is SStotal

, what is SSwithin

, what is SSbetween

, how are they related?

(a) Group 1 Group 2 Group 3

321

534

567

Textbook 17.6 – A psychologist tests whether shy college students initiate more eye contacts with strangers because of training sessions in assertive behavior. Assume the 8 subjects, coded A, B, …, G, H are tested repeatedly after zero, one, two, and three training sessions. The results are expressed as the number of eye contacts:

Subj. Zero One Two Three

A 1 2 4 7

B 0 1 2 6

C 0 2 3 6

D 2 4 6 7

E 3 4 7 9

F 4 6 8 10

G 2 3 5 8

H 1 3 5 7

Tsubj

14

9

11

19

23

28

18

16T

group13 25 40 60 G = 138

Given: SS

between = 154.12

SSwithin

= 132.75SS

total = 286.87

Source SS df MS F

Between 154.12 3 51.37 16.62

Within 132.75 28 -

- Subject 67.87 7 -

- Error 64.88 21 3.09

Total 286.87 31 -

Reject H0 –

Trainings have an effect on number of eye contacts initiated

Source SS df MS F

Between 154.12 3 51.37 16.62

Within 132.75 28 -

- Subject 67.87 7 -

- Error 64.88 21 3.09

Total 286.87 31 -

Effect size:

Textbook 17.6 – A psychologist tests whether shy college students initiate more eye contacts with strangers because of training sessions in assertive behavior. Assume the 8 subjects, coded A, B, …, G, H are tested repeatedly after zero, one, two, and three training sessions. The results are expressed as the number of eye contacts:

Subj. Zero One Two Three

A 1 2 4 7

B 0 1 2 6

C 0 2 3 6

D 2 4 6 7

E 3 4 7 9

F 4 6 8 10

G 2 3 5 8

H 1 3 5 7Xgroup 1.625 3.125 5 7.5

So Group 3 is different from Groups 0, 1, 2.

Group 2 is different from Group 0.

Multiple Comparisons:

ANOVA process:

Textbook 17.6 – A psychologist tests whether shy college students initiate more eye contacts with strangers because of training sessions in assertive behavior. Assume the 8 subjects, coded A, B, …, G, H are tested repeatedly after zero, one, two, and three training sessions. The results are expressed as the number of eye contacts:

Subj. Zero One Two Three

A 1 2 4 7

B 0 1 2 6

C 0 2 3 6

D 2 4 6 7

E 3 4 7 9

F 4 6 8 10

G 2 3 5 8

H 1 3 5 7

Xgroup

1.625 3.125 5 7.5

Example: Size of effect between 0 trainings and 3 trainings.

Effect size:

Could also do effect size between 0 and 2 trainings, 1 and 3 trainings, etc…

Things to look out for:

Alternate hypothesis in AnovaConfidence interval is always taken as per two tailed test tableCalculating HSD(remember to take the proper degrees of freedom for q)

A few more solved questions:

(26.4 – 18.6) – 0 =2.4

3.25=

Alcohol consumption causes an increase in mean performance errors in a driving simulator.

An investigator wishes to determine whether alcohol consumption causes a deterioration in the performance of automobile drivers. Before the driving test, subjects drink a glass of orange juice, which, in the case of the treatment group, is laced with two ounces of vodka. Performance is measured by the number of errors made on a driving simulator. A total of 120 subjects are randomly assigned, in equal numbers, to the two groups. For subjects in the treatment group, the mean number of errors equals 26.4, and for subjects in the control group, the mean number of errors equals 18.6. The estimated standard error equals 2.4.

Interpretation:

H0:

H1:

Decision rule: Reject H0 at the 0.05 level of significance, if t ≥ _____

with ____ degrees of freedom.

Decision:

t test for two independent samples!(directional or non-directional?)

μT – μ

C ≤ 0

μT – μ

C > 0

1.671

118

Reject H0.

An investigator wishes to determine whether alcohol consumption causes a deterioration in the performance of automobile drivers. Before the driving test, subjects drink a glass of orange juice, which, in the case of the treatment group, is laces with two ounces of vodka. Performance is measured by the number of errors made on a driving simulator. A total of 120 subjects are randomly assigned, in equal numbers, to the two groups. For subjects in the treatment group, the mean number of errors equals 26.4, and for subjects in the control group, the mean number of errors equals 18.6. The estimated standard error equals 2.4.

Specify the p-value for this test result.Calculate a 95% confidence interval for the true population mean difference and interpret this interval.

Use Cohen’s d to estimate the effect size, given that the pooled standard deviation s

p equals 13.15. Is this a large, medium, or small effect?

p < 0.001

[3, 12.6](XT – X

C) ± t

conf(s

x1 – x2)

d = X

1 – X

2

√sp

2 = 26.4 – 18.6

13.15 = 0.59 Medium effect

Ex. 1 - A psychologist tests whether a series of workshops on assertive training increases eye contacts initiated by shy college students in controlled interactions with strangers. A total of 32 subjects are randomly assigned, 8 to a group, to attend either 0, 1, 2, or 3 workshop sessions.Use the given information to complete the ANOVA summary table below.

SOURCE SS df MS F

Between 154.12

Within X

Total 286.87 X X132.75

31

3

2851.37

4.74

10.84

Ex. 2 - Students were given different drug treatments before studying for their midterm. Some were given a memory drug, some a placebo drug, and some no treatment. The midterm scores (%) are shown below for the three different groups. At the 0.05 level of significance, do any of the treatments have an effect?

X = 83.4 X = 50 X = 16.6

Not all the means are the same – treatment has some effect.

Ex. 2 - Students were given different drug treatments before studying for their midterm. Some were given a memory drug, some a placebo drug, and some no treatment. The midterm scores (%) are shown below for the three different groups. At the 0.05 level of significance, do any of the treatments have an effect?

417 250 83

SOURCE SS df MS F

BetweenWithin XTotal X X

11,155.6

12,490 14

2

121334.4

5577.8

111.2

50.16

Grand total (G) = 750Group Totals (T)

ANOVA Summary Table

Ex. 2 - Students were given different drug treatments before studying for their midterm. Some were given a memory drug, some a placebo drug, and some no treatment. The midterm scores (%) are shown below for the three different groups. At the 0.05 level of significance, do any of the treatments have an effect?

X = 83.4 X = 50 X = 16.6

- How strong is the effect? - Which means are different?