inferential statistics: hypothesis testing

King Mongkut’s University of Technology North BangkokFaculty of Information Technology

Click to edit Master text stylesSecond level

Third level Fourth level

Fifth level

Click to edit Master title styleKing Mongkut’s University of Technology North BangkokFaculty of Information Technology

ON THE TECHNOLOGICAL FRONTIER WITH ANALYTICAL MIND AND PRACTICE

Inferential Statistics:Hypothesis Testing

Testing Population VariancesAnalysis of Variance – ANOVA


EstimationEstimate population meansEstimate population proportionEstimate population variance

Hypothesis testingTesting population meansTesting categorical data / proportionTesting population variancesHypothesis about many population means

Content


Single population varianceChi-square test

Two populations varianceF-test

Analysis of Variance – test many population meansOne-way ANOVATwo-way ANOVA

Testing Population Variances


AssumptionNormal distribution of population

Test the population variance σ2 from a sample against a specified population variance σ0

2.Hypothesis

Single Population Variance


Test statistic – chi-square

Critical Region

Single Population Variance


A fluorescent lamp factor knows that the lifespan of the lamps is normally distributed with variance of 10,000 hr2. In an inspection, 20 sample lamps are tested and it is found that the variance is 12,000 hr2. Can a conclusion be drawn that the variance of lamp’s lifespan has changed at significant level 0.05?

HypothesisH0: σ2 = 10,000H1: σ2 ≠ 10,000

α = 0.05

Example 1


Calculate test statistic

Degree of freedom = 20 – 1 = 19χ2

(1-0.025),19 = 8.91 , χ2(0.025),19 = 32.85

The calculated chi-square: 8.91 < 22.8 < 32.85, not falling in two-tailed critical region.

Accept H0 and reject H1

The variance of lamp’s lifespan has not changed at significant level 0.05

Example 1

20

22 S)1n(

8.2210000

)12000)(120(2


A company claims that the standard deviation of its thermometer does not exceed 0.5 oC. To verify this, 16 thermometers are sampled. It is found that the standard deviation is 0.7 oC. Is the claim true at significant level 0.01?

HypothesisH0: σ2 ≤ 0.25H1: σ2 > 0.25

α = 0.01

Example 2



Degree of freedom = 16 – 1 = 15χ2

(0.01),15 = 30.58The calculated chi-square: 29.4 < 30.58, not falling in

two-tailed critical region.Accept H0 and reject H1

The claim is true at significant level 0.01

Example 2

20

22 S)1n(


Campare between two variancesSir Ronald Fisher found that

Given s12 and s2

2 are variances of the first and second sample groups of size n1 and n2 respectively,

Both sample groups are randomly selected from normally distributed populations,

The skewness of graph changes corresponding to the degree of freedoms of samples, which are n1-1 and n2-1.

This distribution has become known as Fisher Distribution or F-Distribution

Two Populations Variances


http://en.wikipedia.org/wiki/F-distribution

http://en.wikipedia.org/wiki/F-distribution


F-test is used in model fit such as ANOVA and Linear Regression Analysis in order to determine error (i.e. variance) from the model.

Applicable to 2 populations onlyIf populations are not normally distributed, the test will

be inaccurate.

Application and Limitation



df1 = n1-1 และ df2 = n2-1


Alternate form



The sale count of bicycles in one week of two retailers are as follows

Retailer A: 65 46 57 43 58Retailer B: 52 41 43 47 32 49 57

If the sale count in one week is normally distributed, test if the variances of the sale count the two retailers are different at significant 0.02.

Hypothesis

α = 0.02

Example 1


Calculate variances

SA2 = 82.7, SB

2 = 66.143Calculate test statistic

From H0, σ12 = σ2

2

Degree of freedomdf1 = 5-1 = 4, df2 = 7-1 = 6

Example 1

1NXXS

2

2


Critical region

Accept H0 and reject H1

The variances of the sale count the two retailers are not different at significant 0.02

Example 1


From a previous study, the variance of time required for female workers to assemble a product is less than that of male workers. To re-verify the study, 11 male workers and 14 female workers are sampled. The observed standard deviations of the assembling time are 6.1 for male and 5.3 for female. Assuming normal distribution of assembling time, test if the result of the previous study is accurate at significant level 0.01.

HypothesisH0: σm

2 ≤ σf2

H1: σm2 > σf

2

α = 0.01

Example 2



Degree of freedomdf1 = 11-1 = 10, df2 = 14-1 = 13

Critical region F0.01(10,13) = 4.10Calculated F is 1.325 < 4.10Accept H0 and reject H1

The variance of time required for female workers to assemble a product is not less than that of male workers at significant level 0.01

Example 2


Test if any of multiple means are different from each otherOne-way ANOVA: 1 variables – 3 or more groups

Dependent variable is assumed is of interval or ratio scaleAlso used with ordinal scale data

Can describe the effect of independent variable on dependent variableTwo-way ANOVA: two independent, one dependent variablesMANOVA: Two or more dependent variables

Can describe interaction between two independent variables

Analysis of Variance (ANOVA)


Test the means (of dependent variable) between groups as specified by an independent variable that are organized in 3 or more groups (dichotomous)Occupation: Student, Lecturer, Doctor (1 var - 3 groups)Salary: dependent variable

AssumptionsDependent variable is either an interval or ratio (continuous)Dependent variable is approximately normally distributed for each

category of the independent variableThere is equality of variances between the independent groups

(homogeneity of variances).Independence of cases.

One-way ANOVA


Total Variance = Between-Group Variance + Within-Group VarianceBetween-Group Variance

Describe the difference of means between groups, which is the effect on variable of interest

Within-Group VarianceDescribe the difference of means within each group, which is

the effect caused by other factors, called ErrorH0 : μ1 = μ2 = μ3 = … = μn

H1 : μ1 ≠ μ2 ≠ μ3 ≠ … ≠ μn (at least one different pair)

One-way ANOVA Concept


SST = SSB + SSW

One-way ANOVA TableSource of

Variance Degree of

Freedom (df) Sum Square (SS) Mean Square (MS) F-ratio

Between Groups

(Treatment)

k-1

Within Groups (Error)

n-k

Total n-1

k: number of groups n: number of samples

MSWMSBF

1kSSBMSB

knSSWMSW

nTXSST

ij

n

i

K

j

22

11

K

j j

jn

iij

K

j nT

XSSWj

1

2

1

2

1


Tj: sum of frequencies in each groupT: sum of all frequenciesnj: frequency in each groupk: number of groupxij: the ith data (row) of jth group (column) : the mean of group j : overall mean

One-way ANOVA Table

jX

tX


The survey result of the attitude of the executives in small, medium, and large companies toward management administration is shown in the table. Test if the attitudes of the executives from different company sizes are different at significant level 0.05.

Example 1

AttitudeSmall Medium Large

7 4 107 4 105 2 94 2 67 3 10

Tj 30 15 45 = 906 3 9 = 6

188 49 417 = 654

jX

tX

jn

iij

K

j

X1

2

1


HypothesisHo : 1 = 2 = 3 H1 : 1 ≠ 2 ≠ 3

α = 0.05Calculate test statistic

= (7-6)2 + (7-6)2 + (5-6)2 + (4-6)2 + (7-6)2 + (4-6)2 + (4-6)2 + (2-6)2 + (2-6)2 + (3-6)2 + (10-6)2 + (10-6)2 + (9-6)2 + (6-6)2 + (10-6)2 = 114

Example 1


= 5(6-6)2 + 5(3-6)2 + 5(9-6)2 = 0+45+45 = 90SSW = SST – SSB

= 114 - 90 = 24

= 90 / (3-1) = 45

Example 1

1kSSBMSB kn

SSWMSW

= 24 / (15-3) = 2

MSWMSBF

= 45 /2 = 22.5


Degree of freedom dfB=3-1=2, dfW=15-3=12

F0.05(2,12)= 3.89The calculated F is 22.5 > 3.89Reject H0 and accept H1

The attitudes of the executives from different company sizes are different at significant level 0.05

Example 1Source of Variance

df SS MS F

Between Groups 2 90 45 22.5Within Groups 12 24 2

Total 14 114


One-way ANOVA does not tell which pairs have different means

Post-hoc test (or Post-hoc Analysis or Multiple Comparison) is used to identify the different pairs.

*No need if ANOVA accepts H0 (means are not different)

Post-hoc Test


Methods requiring equality of variances1. Least-Significant Different (LSD) 2. Waller – Duncan3. S-N-K (Student-Newman-Keuls) 4. Dunnett’s C5. Bonferroni 6. Sidak 7. Scheffe 8. R-E-G-WF9. Tukey’s HSD 10. R-E-G-WQ 11. Tukey’s–b 12. Duncan13. Hochberg’s GT2 14. Gabriel

Methods not requiring equality of variances1. Tamhane’s T2 2. Dunnett’s T3 3. Games-Howell 4. Dunnett’s C

Post-hoc Test


Fisher’s Least-Significant Difference proposed by R.A. Fisher to compare multiple pairs at the same time

Calculate LSD

If n1 = n2 then

Compare to LSD valueIf > LSD then the means of the pair are different i ≠ j

Otherwise, the means are not different i = j

Least-Significant Different (LSD)


=0.05, n – k = 15 – 3 = 12, t(0.025, 12) = 2.18

Comparison

At significant level 0.05, the attitude of the executives from small companies is higher than that of the medium ones. And the attitude of the executives from large companies is higher than that of the medium and small ones.

From Example 1

n are equal in the 3 groups

Pair LSD ResultSmall-Medium 6-3=3 1.95 0.05 1 ≠ 2

Small-Large 6-9=-3 1.95 0.05 1 ≠ 3

Medium-Large 3-9=-6 1.95 0.05 2 ≠ 3


Require the sample sizes to be the same

k = number of groups, dfw = n-k, q is obtained from q-table

Compare to HSD valueIf > HSD then the means of the pair are different i ≠ j


Tukey’s Honesty Significant Difference (HSD)


=0.05, k = 3, n – k = 15 – 3 = 12, q0.05, 3, 12 = 3.77

Comparison


From Example 1

n are equal in the 3 groups so HSD is applicable

Pair HSD ResultSmall-Medium 6-3=3 2.38 0.05 1 ≠ 2

Small-Large 6-9=-3 2.38 0.05 1 ≠ 3

Medium-Large 3-9=-6 2.38 0.05 2 ≠ 3


Scheffe or S-Method is applicable to different sample sizes

k = number of groups, dfB= k-1, dfw = n-k, MSW from ANOVA

Compare to S valueIf > S then the means of the pair are different i ≠ j


Scheffe


=0.05, k=3, dfb=2, dfW=12, F0.05(2,12) = 3.88

From Example 1


Comparison


From Example 1Pair S Result

Small-Medium 6-3=3 2.49 0.05 1 ≠ 2 Small-Large 6-9=-3 2.49 0.05 1 ≠ 3

Medium-Large 3-9=-6 2.49 0.05 2 ≠ 3


Four teaching methods are applied to 4 groups of students. Based on the exam scores in the table, test if the four methods give different results at significant level 0.01

Example 2

Method 1 Method 2 Method 3 Method 456737924

118779

6985447

341145

n1 = 8 n2 = 5 n3 = 7 n4 = 6 N = 26T1 = 43 T2 = 42 T3 = 43 T4 = 18 T = 146


HypothesisH0 : 1 = 2 = 3 = 4 H1 : 1 ≠ 2 ≠ 3 ≠ 4

α = 0.01SSB

Example 2


SSW of each group

Example 2


SST = SSB + SSW

One-way ANOVA TableSource of

Variance Degree of

Freedom (df) Sum Square (SS) Mean Square (MS) F-ratio

Between Groups

(Treatment)

k-1

Within Groups (Error)

n-k

Total n-1

k: number of groups n: number of samples

MSWMSBF

1kSSBMSB

knSSWMSW

nTXSST

ij

n

i

K

j

22

11

K

j j

jn

iij

K

j nT

XSSWj

1

2

1

2

1


F0.01(3,22) = 4.82The calculated F is 7.01 > 4.82Reject H0 and accept H1

The four teaching methods give different results at significant level 0.01

Example 2Source of Variance

df SS MS F

Between Groups 3 82.22 27.41 7.01Within Groups 22 85.93 3.91

Total 25 168.15


Use to determine the effect of 2 factors /treatments (independent variables) on one dependent variableOccupation: Student, Lecturer, DoctorAge: less than 20, 20-30, 31-40, 41 or olderSalary: dependent variable

AssumptionsDependent variable is either interval or ratio (continuous)The dependent variable is approximately normally distributed for

each combination of levels of the two independent variablesHomogeneity of variances of the groups formed by the different

combinations of levels of the two independent variables.Independence of cases

Two-way ANOVA


Two-way ANOVA comparesMeans between columnsMeans between rowsMeans from the interaction of factors

Sum Square Row (SSR): variation effect of the 1st factorSum Square Column (SSC): variation effect of the 2nd factorSum Square Row Column (SSRC): variation effect of the

interaction of the two factorsSum Square Error (SSE): Error caused by external factorsSum Square Total (SST) = SSR + SSC + SSRC + SSE

Two-way ANOVA Concept


r: number of rows c: number of columnsn: number of

observationsdf: degree of freedom

Two-way ANOVA Table


Two-way ANOVA Table


Note that the formula on right-handed side assumes equal number of observations in each cellThat’s why n is number of observations

In full form:cn in SSR should be considered as the number of

observations in each rowrn is SSC should be considered as the number of

observations in each columnrcn is total number of observations

Two-way ANOVA Table


An experiment is to determine the effect of loaded weight and speed on fuel consumption (based on distance travelled).

Example 1

Three different speeds and two different loaded weights are tested.

Factor Level Response Speed 70, 90, 110

Km/Hr Total Kilometers

Weight 60, 200 Kg


HypothesisH0 : 1 = 2 = 3 = 4 = 5 = 6

H1 : 1 ≠ 2 ≠ 3 ≠ 4 ≠ 5 ≠ 6


Example 1xi

xj

xij

xt

xijk


F-Critical for factor speed [F0.05(2,12)]= 3.89 F-Critical for factor weight [F0.05(1,12)] = 4.75F-Critical for interaction [F0.05(2,12)] = 3.89


Error VS TotalCompare SSE against SSTRatio between SSE and SST should be less than 1/5. Otherwise, there

exist other factors affecting the experiment. The result may potentially be invalid.

From example SSE/SST: 480 / 48979.78 < 1/5Sum Square of each factor

Speed = 44527.44, weight = 3872Speed is more influencing than weightChanging speed with the same weight affects distance travelled to a

greater extent than changing weight with the same speed

Interpreting Result


Critical regionBoth speed and weight reject H0 meaning both factors

affect the distance travelledInteraction between speed and weight accept H0 meaning

the interaction does not affect distance travelled as both factors separately affect the distance travelled in the same way.

Interpreting Result


A marketing research is to study the effect of packaging and advertising of facial tissue on its sales. Each combination of packaging and advertising is observed 10 time.

Do these two factors affect sales at significant level 0.05?

Example 2Advertising Packaging

Square Rectangle Round With advertising 52 28 15 48 35 14 43 34 23 50 32 21 43 34 14 44 27 20 46 31 21 46 27 16 43 29 20 49 25 14

Sum 464 302 178 944With no advertising 38 43 23 42 34 25 42 33 18 35 42 26 33 41 18 38 37 26 39 37 20 34 40 19 33 36 22 34 35 17

Sum 368 378 214 960Total 832 680 392 1904


HypothesisH0 : 1 = 2 = 3 = 4 = 5 = 6

H1 : 1 ≠ 2 ≠ 3 ≠ 4 ≠ 5 ≠ 6


Example 2


Example 2


ResultPackaging, rejects H0, affecting the salesAdvertising, accepts H0, not affecting the salesInteraction, rejects H0, affecting the sales

The extent the packaging affect the sales is influenced by advertising.

Source of Variance

df SS MS Fcalculate Fcritical

Packaging 2 4,994.10 2,497.05 209.8 F0.05(2,54) =3.17Advertising 1 4.20 4.20 0.35 F0.05(1,54) =4.02Interaction

Effect2 810.20 405.10 34 F0.05(2,54) =3.17

Error 54 643.20 11.90Total 59 6,451.70


http://www.csun.edu/~amarenco/Fcs%20682/When%20to%20use%20what%20test.pdf

A useful resource

inferential statistics: hypothesis testing

Documents

specified population

sample lamps

significant level

calculated chisquare

technological frontier

analytical mind