t-test and anova for balanced datahosting03.snu.ac.kr/~hokim/sas/2013/anovae.pdf · 2016-10-17 ·...

T-test and ANOVA for Balanced Data

Ho Kim

GSPH, SNU

(hosting03.snu.ac.kr/~hokim)

Single Sample Analysis

dataset: peppers Peppers Dataset

Obs angle1 32 113 -74 25 36 87 -38 -29 13

10 411 712 -113 414 715 -116 417 1218 -319 720 521 322 -123 924 -725 226 427 828 -2

proc means data=peppers mean std stderr t probt;

run;

options

1. stderr: the standard error of the mean2. t: 3. probt: the significance probability of the t test

The MEANS Procedure

분석 변수 : angle

평균값 표준편차 표준오차 t값 Pr > |t|

-----------------------------------------------------------------

3.1785714 5.2988718 1.0013926 3.17 0.0037

-------------------------------------------------------------------

0 : 0H

Two Related Samples : paired t-test

dataset: pulse Pulse Dataset

Obs pre post d

1 62 61 1

2 63 62 1

3 58 59 -1

4 64 61 3

5 64 63 1

6 61 58 3

7 68 61 7

8 66 64 2

9 65 62 3

10 67 68 -1

11 69 65 4

12 61 60 1

13 64 65 -1

14 61 63 -2

15 63 62 1

d = pre-post

(difference in rate)

proc means data=pulse mean std stderr t probt;

var d;

run;

The MEANS Procedure

분석 변수 : d

평균값 표준편차 표준오차 t값 Pr > |t|-------------------------------------------------------------------

1.4666667 2.3258383 0.6005289 2.44 0.0285-------------------------------------------------------------------

Two-sided p-value

One-sided p-value=0.0285/2=0.0143 for vs.0 : 0H d 1 : 0H d

Two Independent Samples

dataset: bullets Bullets Dataset

Obs powder velocity

1 1 27.3

2 1 28.1

3 1 27.4

4 1 27.7

5 1 28.0

6 1 28.1

7 1 27.4

8 1 27.1

9 2 28.3

10 2 27.9

11 2 28.1

12 2 28.3

13 2 27.9

14 2 27.6

15 2 28.5

16 2 27.9

17 2 28.4

18 2 27.7

proc ttest data=bullets;

var velocity;class powder;

run;

The TTEST Procedure

Lower CL Upper CL Lower CL

Variable powder N Mean Mean Mean Std Dev

velocity 1 8 27.309 27.638 27.966 0.2596

velocity 2 10 27.841 28.06 28.279 0.2106

velocity Diff (1-2) -0.771 -0.422 -0.074 0.2582

Upper CL

Variable powder Std Dev Std Dev Std Err Minimum Maximum

velocity 1 0.3926 0.799 0.1388 27.1 28.1

velocity 2 0.3062 0.5591 0.0968 27.6 28.5

velocity Diff (1-2) 0.3467 0.5276 0.1644

Variable Method Variances DF t Value Pr > |t|

velocity Pooled Equal 16 -2.57 0.0206

velocity Satterthwaite Unequal 13.1 -2.50 0.0267

Equality of Variances

Variable Method Num DF Den DF F Value Pr > F

velocity Folded F 7 9 1.64 0.4782

For H0: Variances are equal, F = 1.64 DF = (7,9)

Multiple Comparisons and Pre-planed Comparisons

F-test: ‘All effects are zero.’

We still don’t know which effects are not zero even after rejecting Ho.

ex)

if is rejected, which is large and which is small ?

Multiple Comparisons

Several methods

LSD (least significant difference): Perform t tests for all possible pairs.

Duncan’s multiple range test: Compare the difference of means to the

pre-fixed values -> determine equality of means.

Inflating type I error is a major problem for multiple comparisons.

0 :H

0 1 2 3 4: 0H

0H i i

ex) let’s assume that is the significance level of a test.

multiple comparison for

For k=4,

overall = 0.1855, inflated type I error.

01 1 01 01

02 2 02 02

0 0 0 01 02

01

: 0 ( ) 1

: 0 ( ) 1

( ) where and

(

Let H p do not reject H H is true

H p do not reject H H is true

then p do not reject H H H H H

p do not reject H and do not reje

02 0

2

)

(1- ) (1- ) (1- )

ct H H

1 2 3 k

4

(1 ) (1 )

1 0.1855 0.8145 ( .95) .95

k

Bonferroni Correction : For m multiple comparisons,

individual significance level =

then over-all significance level becomes

Ex1. )m= 4

Ex2. ) If we have 10 hypothesis, perform individual test with

p= . Then over-all p=0.05 for the whole tests.

This is called “Bonferroni corrected p-value”.

m

40.05(1 ) 0.95 1 0.05

4

0.050.005

10

Multiple comparisons is performed after collecting data if

there were no planned hypotheses.

In many cases, we have preplanned comparisons when we

design a study. This is called preplanned comparisons.

We can do this by using CONTRAST, ESTIMATE, LSMEAN, etc

in SAS.

We don’t need to worry about type I error problems for

preplanned comparisons.

It is very important to learn how to use contrast statement.

BRAND Data

Explanatory var: brand

response var: wear

Preplanned Comparisons

model

i ij

i ij

ijy

ACMEiAJAXCHAMPTUFFYXTRA

Mean of i-th level

Veneer data set

Obs brand wear

1 ACME 2.3

2 ACME 2.1

3 ACME 2.4

4 ACME 2.5

5 CHAMP 2.2

6 CHAMP 2.3

7 CHAMP 2.4

8 CHAMP 2.6

9 AJAX 2.2

10 AJAX 2.0

11 AJAX 1.9

12 AJAX 2.1

13 TUFFY 2.4

14 TUFFY 2.7

15 TUFFY 2.616 TUFFY 2.7

17 XTRA 2.3

18 XTRA 2.5

19 XTRA 2.3

20 XTRA 2.4

Overall mean

Effect of i-th level

US vs Foreign

• Simultaneous testing

• estimate

0

1 1

3 2

1 1

3 2

: ( ) ( )

( ) ( ) 0

.333 .333 .333 - .5 - .5

2 ( ) 3 ( ) 0

ACME AJAX CHAMP TUFFY XTRA



H

2 2 2 - 3 - 3

Whole population is divided into homogeneous blocks.

We do randomization in each block.

We are not interested in the Block effects. We are interested in

the effect of the other factor.

ex) pesticide data

BLOCK=location: 1, 2, 3

5 kinds of BLEND: A, B, C, D, E

Interested in BLEND effects after controlling for BLOCK

effect.

Randomized-Blocks Design

2 factors

Pesticide Dataset

Obs block blend pctloss

1 1 B 18.2

2 1 A 16.9

3 1 C 17.0

4 1 E 18.3

5 1 D 15.1

6 2 A 16.5

7 2 E 18.3

8 2 B 19.2

9 2 C 18.1

10 2 D 16.0

11 3 B 17.1

12 3 D 17.8

13 3 C 17.3

14 3 E 19.8

15 3 A 17.5

ANOVA for Randomized-Blocks Design

proc anova data=pestcide;

class block blend;

model pctloss=block blend;

run;

The ANOVA ProcedureDependent Variable: pctloss

Sum ofSource DF Squares Mean Square F Value Pr > F

Model 6 13.20400000 2.20066667 2.52 0.1133Error 8 6.99200000 0.87400000Corrected Total 14 20.19600000

R-Square Coeff Var Root MSE pctloss Mean0.653793 5.329987 0.934880 17.54000

Source DF Anova SS Mean Square F Value Pr > F

block 2 1.64800000 0.82400000 0.94 0.4289blend 4 11.55600000 2.88900000 3.31 0.0705

3 factors: run position mat

Latin Square Design

Position

run 1 2 3 4

1 B D A C

2 D B C A

3 A C B D

4 C A D B

MAT: materials (A,B,C,D)

POS: (1, 2, 3, 4)

RUN: 4 level

WTLOSS: weight loss

SHRINK:

fewer # of repeats -> less expensive.

limit: Interactions can not be tested.

Garments Dataset

Obs run pos mat wtloss shrink

1 2 4 A 251 50

2 2 2 B 241 48

3 2 1 D 227 45

4 2 3 C 229 45

5 3 4 D 234 46

6 3 2 C 273 54

7 3 1 A 274 55

8 3 3 B 226 43

9 1 4 C 235 45

10 1 2 D 236 46

11 1 1 B 218 43

12 1 3 A 268 51

13 4 4 B 195 39

14 4 2 A 270 52

15 4 1 C 230 48

16 4 3 D 225 44

Analysis of ANOVA for Latin Square Design

proc anova data=garments; class run pos mat;

model wtloss shrink = run pos mat;

run;

The ANOVA Procedure

Dependent Variable: wtloss

Sum of

Source DF Squares Mean Square F Value Pr > F

Model 9 7076.500000 786.277778 12.84 0.0028

Error 6 367.500000 61.250000

Corrected Total 15 7444.000000

R-Square Coeff Var Root MSE wtloss Mean

0.950631 3.267740 7.826238 239.5000


run 3 986.500000 328.833333 5.37 0.0390

pos 3 1468.500000 489.500000 7.99 0.0162

mat 3 4621.500000 1540.500000 25.15 0.0008

The ANOVA Procedure

Dependent Variable: shrink

Sum of


Model 9 265.7500000 29.5277778 9.84 0.0058

Error 6 18.0000000 3.0000000


R-Square Coeff Var Root MSE shrink Mean

0.936564 3.675439 1.732051 47.12500


run 3 33.2500000 11.0833333 3.69 0.0813

pos 3 60.2500000 20.0833333 6.69 0.0242

mat 3 172.2500000 57.4166667 19.14 0.0018

ANOVA for One-Way Classification

proc anova data=veneer; class brand;

model wear=brand; run;

The ANOVA Procedure

Dependent Variable: wear

Sum of


Model 4 0.61700000 0.15425000 7.40 0.0017

Error 15 0.31250000 0.02083333


R-Square Coeff Var Root MSE wear Mean

0.663798 6.155120 0.144338 2.345000


brand 4 0.61700000 0.15425000 7.40 0.0017

Least Significant Difference Comparisons of BRAND Mean

proc anova data=veneer;

class brand;

model wear=brand;

means brand/lsd;

run;The ANOVA Procedure

t Tests (LSD) for wear

노트: This test controls the Type I comparisonwise error rate, not the

experimentwise error rate.

Alpha 0.05Error Degrees of Freedom 15Error Mean Square 0.020833Critical Value of t 2.13145Least Significant Difference 0.2175

Means with the same letter are not significantly different.

T Grouping Mean N brandA 2.6000 4 TUFFYB 2.3750 4 XTRABB 2.3750 4 CHAMPBB 2.3250 4 ACMEC 2.0500 4 AJAX

Confidence Interval for BRAND Means

proc anova data=veneer; class brand;

model wear=brand;

means brand/lsd clm; run;

The ANOVA Procedure

t Confidence Intervals for wear

Alpha 0.05

Error Degrees of Freedom 15

Error Mean Square 0.020833

Critical Value of t 2.13145

Half Width of Confidence Interval 0.153824

brand N Mean 95% Confidence Limits

TUFFY 4 2.60000 2.44618 2.75382

XTRA 4 2.37500 2.22118 2.52882

CHAMP 4 2.37500 2.22118 2.52882

ACME 4 2.32500 2.17118 2.47882

AJAX 4 2.05000 1.89618 2.20382

US vs Foreign

• We need to input the coefficients of the model parameters

at intercept , brand in SAS

0

1 1

3 2

1 1

3 2

: ( ) ( )

( ) ( ) 0

.333 .333 .333 - .5 - .5

2 ( ) 3 ( ) 0




H

2 2 2 - 3 - 3

1 2 3 4 5, , , , ,

• Intercept Brand1 Brand2 Brand3 Brand4 Brand5

• 0 0.333 0.333 0.333 -0.5 -0.5

• 0 2 2 2 -3 -3

( )i i iE Y

i ij

i ij

ijy

ACMEiAJAXCHAMPTUFFYXTRA

Mean of i-th level

1 2 3 1 2 3

4 5 4 5

1 1

3 3

1 1

2 2

( ) ( ) (1)

( ) ( ) (2)

Ho: (1)-(2)=0

Simultaneous Contrasts among US BRAND Means

proc glm data=veneer;

class brand;

model wear=brand;

contrast 'US BRANDS' brand 1 -1 0 0 0, brand 1 0 -1 0 0;

run;

The GLM ProcedureDependent Variable: wear

Sum ofContrast DF Contrast SS Mean Square F Value Pr > F

US BRANDS 2 0.24500000 0.12250000 5.88 0.0130

1 2 3

1 2 1 3

1 2 1 3

1 2 1 3

: ( )

: and

=0 and 0

( ) 0 and ( ) 0

Ho

Ho

Int Brand1 Brand2 Brand3 Brand4 Brand5

0 1 -1 0 0 0 and

0 1 0 -1 0 0

Are the means of ACME AJAX CHAMP all the same ?

ANOVA and Contrast with PROC GLM

proc glm data=veneer; class brand;

model wear=brand;

contrast 'ACME vs AJAX' brand 1 -1 0 0 0;

run;The GLM Procedure

Dependent Variable: wearSum of


Model 4 0.61700000 0.15425000 7.40 0.0017Error 15 0.31250000 0.02083333Corrected Total 19 0.92950000

R-Square Coeff Var Root MSE wear Mean0.663798 6.155120 0.144338 2.345000

Source DF Type I SS Mean Square F Value Pr > Fbrand 4 0.61700000 0.15425000 7.40 0.0017

Source DF Type III SS Mean Square F Value Pr > Fbrand 4 0.61700000 0.15425000 7.40 0.0017

Contrast DF Contrast SS Mean Square F Value Pr > FACME vs AJAX 1 0.15125000 0.15125000 7.26 0.0166

Contrasts among BRAND Means


model wear=brand;

contrast 'US vs FOREIGN' brand 2 2 2 -3 -3;

contrast 'A-L vs C-L' brand 1 1 -2 0 0;

contrast 'ACME vs AJAX' brand 1 -1 0 0 0;

contrast 'TUFFY vs XTRA' brand 0 0 0 1 -1;

run;The GLM Procedure


Contrast DF Contrast SS Mean Square F Value Pr > F

US vs FOREIGN 1 0.27075000 0.27075000 13.00 0.0026

A-L vs C-L 1 0.09375000 0.09375000 4.50 0.0510

ACME vs AJAX 1 0.15125000 0.15125000 7.26 0.0166

TUFFY vs XTRA 1 0.10125000 0.10125000 4.86 0.0435

1 2 3 1 2 3

1 2 3

1: ( ) ---> 0.5 0.5 - 0

2

0.5 0.5 0.5 0.5 ( ) 0

Ho

Estimating Difference between BRANDE mean


model wear=brand;

estimate 'ACME vs AJAX' brand 1 -1 0 0 0;

run;

The GLM Procedure


Standard

Parameter Estimate Error t Value Pr > |t|

ACME vs AJAX 0.27500000 0.10206207 2.69 0.0166

Estimating Mean of US Brand


model wear=brand;

estimate 'US MEAN' intercept 3 brand 1 1 1 0 0/divisor=3;

run;

The GLM Procedure


Standard


US MEAN 2.25000000 0.04166667 54.00 <.0001

1 2 3 1 2 3

1 2 3

1 1( ) --> ( )

3 3

1 1 1

3 3 3

ACME - AJAX


model wear=brand;

estimate 'Acme-Ajax' brand 1 -1 0 0 0;

run;

or

proc ttest data=veneer;

class brand;

var wear;

where brand in ('ACME', 'AJAX') ;

run;

Standard


Acme-Ajax 0.27500000 0.10206207 2.69 0.0166

The TTEST Procedure

Lower CL Upper CL Lower CL

Variable brand N Mean Mean Mean Std Dev

wear ACME 4 2.0532 2.325 2.5968 0.0967

wear AJAX 4 1.8446 2.05 2.2554 0.0731

wear Diff (1-2) 0.0131 0.275 0.5369 0.0975

Statistics

Upper CL

Variable brand Std Dev Std Dev Std Err Minimum Maximum

wear ACME 0.1708 0.6368 0.0854 2.1 2.5

wear AJAX 0.1291 0.4814 0.0645 1.9 2.2

wear Diff (1-2) 0.1514 0.3334 0.107

T-Tests

Variable Method Variances DF t Value Pr > |t|

wear Pooled Equal 6 2.57 0.0424

wear Satterthwaite Unequal 5.58 2.57 0.0452

Equality of Variances

Variable Method Num DF Den DF F Value Pr > F

wear Folded F 3 3 1.75 0.6571

Homework

Perform ANOVA using estimate and contrast statements in this file.

Explain the meanings 1) using parameters of the ANOVA model, 2) in words (practical and realistic meaning).

t-test and anova for balanced datahosting03.snu.ac.kr/~hokim/sas/2013/anovae.pdf · 2016-10-17 ·...

Documents