topic 5: parametric analyses - uppsala university

QM STEM Ed 20181

Quantitative Methods in STEM

Education Research

Topic 5: Parametric analyses

Judy Sheard

Faculty of Information Technology

Monash University, Australia

[email protected]

QM STEM Ed 20182

Overview of topic 5

Parametric tests

Assumptions

t-distribution

Degrees of freedom

t-test

One and two tailed tests

ANOVA

Post hoc analysis

QM STEM Ed 20183

Common parametric statistical

tests

Statistical test Hypothesis tested

t-test

(independent-

samples)

There is no difference in the mean scores from two

samples.

t-test (paired-

samples)

There is no difference in the means of two related

measures on a sample or a sample of matched pairs

of subjects.

ANOVA (one-

way)

There is no difference in the means of scores of two

or more samples. Single independent variable.

ANOVA (two-

way)

There is no difference in the means of scores of two

or more samples. Two independent variables are

included and a hypothesis for their interaction

QM STEM Ed 20184

Parametric analysis - assumptions

Measurement of the data on the interval or ratio scale.

Scores are ‘independent’.

Scores are selected from a normally distributed population. (If the sample is large this is not important.)

Homogeneity of variance – if two or more groups are studied, they must come from populations with similar dispersions in their distribution.

QM STEM Ed 20185

Degrees of freedom

The degrees of freedom (df) is defined as the

number of ways in which the data is free to vary.

This is determined by subtracting the number of

restrictions placed on the data from the number of

scores.

For example, a mean is computed by summing n

scores, when n-1 scores have been totaled the nth

score is uniquely determined as it provides the

remainder for the sum. In this case the df is n-1

QM STEM Ed 20186

Another distribution

The central limit theorem states that the sampling distribution of a statistic (e.g. a sample mean) will form a normal distribution, for a sufficiently large sample size.

If we know the standard deviation of the population, we can compute a z-score, and use the normal distribution to evaluate probabilities with the sample mean.

However, for small sample sizes or if we do not know the standard deviation of the population, the distribution of the t statistic (also known as the t-score) is used:

The distribution of the t statistic is called the t distribution or the Student t distribution.

xt

s n

where x bar is the sample mean, μ is the population mean, s is

the standard deviation of the sample, and n is the sample size

(this is for a one sample t-test)

QM STEM Ed 20187

The t distribution

The t-distribution is actually a family of distributions, with a different t distribution for each degree of freedom.

The smaller the sample, and the fewer the degrees of freedom, the flatter (more spread) the t-distribution.

For large sample sizes (around 100), the normal and t-distributions are almost identical.

The t-distribution rather than the normal distribution is more appropriate to use when sample sizes are small and the sample standard deviation is estimated from the population standard deviation.

QM STEM Ed 20188

William Sealy Gosset, who

developed the "t-statistic" and

published it in 1908 under the

pseudonym of "Student“

(https://en.wikipedia.org/)

QM STEM Ed 20189

The t and normal distributions

Comparison of the

t-distribution with 4

degrees of freedom to

the normal distribution.

QM STEM Ed 201810

The t and normal distributions

4 2 2 4

0.1

0.2

0.3

0.4

t distribution with 1 degrees of freedom .

4 2 2 4

0.1

0.2

0.3

0.4


4 2 2 4

0.1

0.2

0.3

0.4


4 2 2 4

0.1

0.2

0.3

0.4


The red curve is the standard normal curve

QM STEM Ed 201811

t-test

One of the most commonly used statistical

tests.

Used to determine whether the means of two

groups differ to a statistically significant

degree.

There are different versions of the t-test.

QM STEM Ed 201812

Which t-test to use?

Independent samples t-test – the mean score from a group is tested against the mean score from another group. The hypothesis tested is:

H0 : μ1 = μ2

HA : μ1 ≠ μ2

Paired or dependent samples t-test – the mean score of a group on one measure is tested against the mean score on another measure.

H0 : μd = 0

HA : μd ≠ 0

One sample t-test – a mean score is tested against a particular value.H0 : μ1 = value

HA : μ1 ≠ value

QM STEM Ed 201813

Assumptions: interval scale

Dependent variables should be on interval or

ratio scale.

There is some debate as to whether ordinal

scale variables should be used for parametric

tests.

Often in educational research (and other

research) parametric analysis is used on

Likert scale variables.

QM STEM Ed 201814

Assumptions: independence

The value of one observation should not

influence the value of another observation –

except in paired samples tests.

For tests between groups – subjects should

appear in only one group.

Examples of violations:

Subject is unwittingly tested twice

Subjects are not randomly assigned to groups.

QM STEM Ed 201815

Assumptions: normality

t-tests assume that the dependent variable is

normally distributed in each population. In practice

we test the distribution of scores of the sample.

Violations do not greatly influence the test,

particularly when:

The sample size is large

The test is two-tailed

The distribution is not especially skewed

Normality of distributions can be tested in SPSS.

QM STEM Ed 201816

Assumptions: homogeneity of

variance

t-tests assume equal variances.

An adjustment can be made to the t-value

when variances are not homogenous.

Levene’s test is used to test for homogeneity

of variances.

SPSS provides a test for homogeneity of

variance and a adjusted t-value.

QM STEM Ed 201817

Anxiety and Exams study

QM STEM Ed 201818

Independent-samples t-test example

Anxiety and Exams study

We would like to see if there are any differences between the male and female students for: assignment marks

exam marks

number of hours spent on exam preparation.

To do this we will compare the mean values of these variables for the males and females using independent-samples t-tests. In each case we are testing the null hypothesis of no difference between the groups.

H0 : μ1 = μ2

HA : μ1 ≠ μ2

QM STEM Ed 201819

Independent samples t-test

We will go through these steps:

Check assumptions

Perform tests

Examine output

Check more assumptions

Report our findings

QM STEM Ed 201820

Assumptions

Assumptions for an independent samples t-test:

Interval scale. √

Independence of observations √

and groups. √

Normality – we will test this in SPSS.

Homogeneity of variance – we will test this in

SPSS.

QM STEM Ed 201821

Investigate descriptives

1. Select the Analyze menu.

2. Click on Descriptive Statistics.

3. Select the variables assignment, exam and

hours and move into the Variable(s) box.

4. Click on Options… to open the Options dialogue box.

5. Click on the Kurtosis and Skewness check boxes.

6. Click on Continue, then OK.

QM STEM Ed 201822

Examine descriptives

Descriptive Statistics

25 1 10 6.32 2.688 -.301 .464 -.801 .902

25 1 9 5.48 2.330 -.574 .464 -.517 .902

25 2 19 9.68 4.404 -.025 .464 -.342 .902

25

Assignment Mark

Exam Mark

Hours of Exam

Preparat ion

Valid N (listwise)

Stat ist ic Stat ist ic Stat ist ic Stat ist ic Stat ist ic Stat ist ic Std. Error Stat ist ic Std. Error

N Minimum Maximum Mean Std.

DeviationSkewness Kurtosis

QM STEM Ed 201823

Examine descriptives


25 1 10 6.32 2.688 -.301 .464 -.801 .902

25 1 9 5.48 2.330 -.574 .464 -.517 .902

25 2 19 9.68 4.404 -.025 .464 -.342 .902

25

Assignment Mark

Exam Mark

Hours of Exam

Preparat ion

Valid N (listwise)

Stat ist ic Stat ist ic Stat ist ic Stat ist ic Stat ist ic Stat ist ic Std. Error Stat ist ic Std. Error



The skewness is

low for all three

distributions

The kurtosis is

low for all three

distributions

QM STEM Ed 201824

Kolmogorov-Smirnov test

The Kolmogorov-

Smirnov test is used to

test that the data comes

from a particular

distribution – in this

case we will test for a

normal distribution.

1. Select Analyze Nonparametric Tests 1-Sample K-S…

2. Select the variables assignment, exam and hours and move into the

Test Variable List box.

3. Check that Normal is selected. This is usually the default.

QM STEM Ed 201825

Kolmogorov-Smirnov test

One-Sample Kolmogorov-Smirnov Test

25 25 25

6.32 5.48 9.68

2.688 2.330 4.404

.120 .228 .169

.088 .097 .136

-.120 -.228 -.169

.599 1.142 .845

.865 .148 .473

N

Mean

Std. Dev iat ion

Normal Parametersa,b

Absolute

Positive

Negativ e

Most Extreme

Dif f erences

Kolmogorov-Smirnov Z

Asy mp. Sig. (2-tailed)

Assignment

Mark Exam Mark

Hours of

Exam

Preparat ion

Test distribution is Normal.a.

Calculated f rom data.b. Since these p-values are all greater than 0.05 we can

conclude that the data has a normal distribution

QM STEM Ed 201826

Independent-samples t-test

1. Select the Analyze Compare Means Independent-Samples T-Test…

2. Select the variables assignment, exam and hours and move into the Test

Variable(s) list box.

3. Select the variable success and move into the Grouping Variable box.

4. Click on Define Groups…

QM STEM Ed 201827


1. Enter the values for the grouping variable. In this case it will be 1

for male and 2 for female

2. Click on Continue, then OK.

3. Another option in the Independent-samples T-Test dialogue

box is to click on Paste and save the commands you have

specified to a script file.

QM STEM Ed 201828

Script file in SPSS

You may work directly from the SPSS Syntax Editor rather than

the GUI interface. You can specify and run all tests from here.

QM STEM Ed 201829


– descriptives

Group Statistics

16 6.13 2.964 .741

9 6.67 2.236 .745

16 4.63 2.446 .612

9 7.00 1.000 .333

16 8.50 4.662 1.165

9 11.78 3.114 1.038

Gendermale

f emale

male

f emale

male

f emale

Assignment Mark

Exam Mark

Hours of Exam

Preparat ion

N Mean Std. Dev iat ion

Std. Error

Mean

QM STEM Ed 201830


– test output

Independent Samples Test

1.258 .274 -.476 23 .639 -.542 1.139 -2.897 1.814

-.515 20.793 .612 -.542 1.051 -2.729 1.645

9.447 .005 -2.765 23 .011 -2.375 .859 -4.152 -.598

-3.410 21.654 .003 -2.375 .696 -3.821 -.929

4.798 .039 -1.878 23 .073 -3.278 1.745 -6.888 .333

-2.100 22.130 .047 -3.278 1.561 -6.513 -.042

Equal variances

assumed

Equal variances

not assumed

Equal variances

assumed

Equal variances

not assumed

Equal variances

assumed

Equal variances

not assumed

Assignment Mark

Exam Mark

Hours of Exam

Preparat ion

F Sig.

Levene's Test f or

Equality of Variances

t df Sig. (2-tailed)

Mean

Dif f erence

Std. Error

Dif f erence Lower Upper

95% Conf idence

Interv al of the

Dif f erence

t-test for Equality of Means

QM STEM Ed 201831

One-tailed or two-tailed?

Hypotheses may be stated in directional or non-

directional form.

Directional means that you predict the direction of

difference before analysis. E.g.

H0 : μ1 > μ2 or HA : μ1 < μ2

It is easier to reject the null hypothesis in the chosen

direction (larger p-value) – but you then must ignore

differences in the other direction.

When in doubt about the direction of change, use a

two-tailed test.

QM STEM Ed 201832


– test output

Independent Samples Test

1.258 .274 -.476 23 .639 -.542 1.139 -2.897 1.814

-.515 20.793 .612 -.542 1.051 -2.729 1.645

9.447 .005 -2.765 23 .011 -2.375 .859 -4.152 -.598

-3.410 21.654 .003 -2.375 .696 -3.821 -.929

4.798 .039 -1.878 23 .073 -3.278 1.745 -6.888 .333

-2.100 22.130 .047 -3.278 1.561 -6.513 -.042

Equal variances

assumed

Equal variances

not assumed

Equal variances

assumed

Equal variances

not assumed

Equal variances

assumed

Equal variances

not assumed

Assignment Mark

Exam Mark

Hours of Exam

Preparat ion

F Sig.

Levene's Test f or

Equality of Variances


Mean

Dif f erence

Std. Error


95% Conf idence

Interv al of the

Dif f erence

t-test for Equality of Means

We are 95% confident that the mean

difference for each population falls between

the upper and lower confidence limits

QM STEM Ed 201833

Homogeneity of variance –

Levene’s test

Levene’s test is used to determine whether the variances are similar (nearly equal). We do this by looking at the p-values.

In our tests the p-value is > 0.05 for the first test and therefore we can conclude that there is no difference in variances between the assignment marks for the male and female groups.

However, for the exam marks and the number of hours in exam preparation, the p-values are far less than 0.05 indicating differences in variances.

This affects the selection of the t-value, df and significance level when reporting results.

QM STEM Ed 201834

Independent-samples t-test results

We now report the results of our tests:

Independent-samples t-tests (n=25) were conducted to evaluate the hypotheses that males and females differ in their marks for assignments and exams and in their preparation time for exams.

No difference was found in the assignment marks between the male and female students. However, the female students scored higher marks (M=7.0, SD=1.0) than males (M=4.6,SD=2.5) in the exam (t(21.7)= -3.41, p<0.05) and the female students spent longer hours (M=11.8,SD=3.1) than the males (M=8.5,SD=4.7) on their work (t(22.1) = -2.10, p<0.05). These differences were significant.

QM STEM Ed 201835

Paired samples t-test example

We would like to see if there is any difference between the assignment and exam marks for the students.

As these are related measures, we will compare the mean values for the assignment and exam marks using paired samples t-tests. In this case we are testing the null hypothesis of no difference between the measures.

H0 : μd = 0

HA : μd ≠ 0

QM STEM Ed 201836

Assumptions

Assumptions for an independent samples t-test:

Interval scale √


Normality of population difference scores –

the difference between the scores for each

subject should be normally distributed. Not

really important for samples greater than 30,

but as our sample is 25 we will check this.

QM STEM Ed 201837

Checking normality of differences


25

-.8400

2.86764

.198

.161

-.198

.989

.282

N

Mean

Std. Dev iation

Normal Parameters a,b

Absolute

Positive

Negative

Most Extreme

Dif f erences



Dif f between

exam and

assignment


Calculated f rom data.b. Since this p-values is greater than 0.05 we can

conclude that the data has a normal distribution

QM STEM Ed 201838

Paired samples t-test

1. Select Analyze Compare Means Paired-Samples T-Test…

2. Select assignment and exam and move into the Paired Variables list box.

3. Select OK.

QM STEM Ed 201839

Paired-samples t-test

– descriptives

Paired Samples Statistics

6.32 25 2.688 .538

5.48 25 2.330 .466

Assignment Mark

Exam Mark

Pair

1

Mean N Std. Dev iation

Std. Error

Mean

Paired Samples Correlations

25 .354 .083Assignment Mark

& Exam Mark

Pair

1

N Correlation Sig.

SPSS also does a correlation

with a Paired samples t-test

QM STEM Ed 201840

Paired-samples t-test

– test output

Paired Samples Test

.840 2.868 .574 -.344 2.024 1.465 24 .156Assignment Mark

- Exam Mark

Pair

1

Mean Std. Dev iat ion

Std. Error

Mean Lower Upper

95% Conf idence

Interv al of the

Dif f erence

Paired Dif f erences


QM STEM Ed 201841

Paired samples t-test results

We now report the results of our test.

A paired-samples t-test (n=25) was conducted to

evaluate the hypotheses that the students’

marks for assignments and exams were

different.

No difference was found between the

assignment marks and exam marks for the

students in this study.

QM STEM Ed 201842

One-sample t-test

We would like to see if there is any difference between the final results achieved by this group of students and the results from other years. The mean result was 48 for the previous five years.

In this case, we have one set of data that we wish to test against a ‘population’ mean, so a one-sample t-test is appropriate to use. We are testing the null hypothesis of no difference between the groups and a particular value.

H0 : μ1 = 48

HA : μ1 ≠ 48

QM STEM Ed 201843

One-sample t-test

1. Select Analyze Compare Means One-Sample T-Test…

2. Select final and move into the Test Variable(s) list box.

3. Enter 48 into the Test Value box.

4. Select OK.

QM STEM Ed 201844

One-sample t-test – output

One-Sample Statistics

25 57.32 20.591 4.118Final Mark

N Mean Std. Dev iat ion

Std. Error

Mean

One-Sample Test

2.263 24 .033 9.320 .82 17.82Final Mark


Mean


95% Conf idence

Interv al of the

Dif f erence

Test Value = 48

QM STEM Ed 201845

One-sample t-test results


A one-sample t-test (n=25) was conducted to evaluate the hypotheses that the final marks of this group of students is different to the mean mark obtained over the previous five years.

The final result of the students (M=57.3, SD=20.6) was higher than the mean of 48 recorded for the previous five years and this difference was significant.

QM STEM Ed 201846

T-106.6500 Quantitative Methods in

Computing/Engineering Education

Research

Topic 5 (part 2): Parametric analyses

Judy Sheard

Faculty of Information Technology

Monash University, Australia

[email protected]

QM STEM Ed 201847

ANOVA

If we wish to make comparisons between more than two

groups, then the appropriate test to use is an analysis

of variance (ANOVA).

There are a number of different versions of ANOVA

depending on:

the number of factors (independent variables);

the number of levels of each independent variable; and

whether the independent variables are unrelated (each group

comprises different subjects) or related (same subjects are

used in each group).

QM STEM Ed 201848

ANOVA varieties

The common term for an ANOVA with 1 factor is one-way ANOVA, with 2 factors we use two-way ANOVA, 3 factors we use three-way ANOVA, and so on...

If we want to also specify the levels within each factor, then we use the following:

An ANOVA with one factor with 3 levels and another factor with 5 levels is called a 3 x 5 ANOVA.

An ANOVA with three factors each with 4 levels is called a 4 x 4 x 4 ANOVA.

QM STEM Ed 201849

ANOVA

One-way ANOVA is used to test that the means scores of two or

more groups are not significantly different.

H0 : μ1 = μ2 = μ3 = … = μk

HA : μ1 ≠ μ2 for at least one pair.

The null hypothesis in ANOVA is tested by comparing two estimates

of variance called mean squares. It compares the variance

between groups to the variance within groups.

The ratio of these variances has as its sampling distribution the F-

distribution, determined by two degrees of freedom values. These

are a between groups df and a within groups df.

F-distributions for different degrees of

freedom

QM STEM Ed 201850

http://www.vosesoftware.com/ModelRiskHe

lp/index.htm

F-distribution

QM STEM Ed 201851

QM STEM Ed 201852

One-way ANOVAs

One-way ANOVAs have one factor (independent variable). There are two main types:

One-way between groups – each group comprises different subjects.

One-way repeated measures – the same subjects are used in each group.

For example: You plan a study of the effect of tutorial tasks on final grades in an introductory programming course.

A study that compared performance of 4 tutorial groups in the final exam would be one-way between groups ANOVA.

A study that compared tutorial performance on 3 successive tests would be a one-way repeated measures ANOVA.

QM STEM Ed 201853

One-way ANOVA example

Performance studyAs part of a study of the first year of a new multi-campus IT degree,

data of student performance was collected. We wish to investigate

whether there is any difference in the performance of students in the

computer systems and programming courses across the five

campuses of the university.

In each case we wish to compare the mean results of five campuses. Since have one factor with five levels, will use a one-way between groups AVOVA.

H0 : μ1 = μ2 = μ3 = … = μk

HA : μ1 ≠ μ2 for at least one pair.

QM STEM Ed 201854

Assumptions

The assumptions for an ANOVA are similar to

those for t-tests.

Interval scale √


Normality of populations – we will test this in

SPSS.

Homogeneity of variance – we will test this in

SPSS.

QM STEM Ed 201855

Descriptives


119 7.00 97.00 63.1681 18.74283 -1.240 .222 1.569 .440

119 16.00 100.00 69.4118 21.30585 -.589 .222 -.347 .440

119

Computer Sy stems

Programming

Valid N (listwise)

Stat istic Stat istic Stat istic Stat istic Stat istic Stat istic Std. Error Stat istic Std. Error



Note the large skewness and

kurtosis value for the Computer

Systems variable.

QM STEM Ed 201856

Normality check


119 119

63.1681 69.4118

18.74283 21.30585

.182 .086

.072 .078

-.182 -.086

1.981 .938

.001 .342

N

Mean

Std. Dev iat ion

Normal Parametersa,b

Absolute

Positive

Negative

Most Extreme

Dif f erences



Computer

Sy stems Programming


Calculated f rom data.b.

The low p-value for computer systems

(less than 0.05) indicates that the data

does not have a normal distribution.

QM STEM Ed 201857

Distribution of results for

Computer Systems

QM STEM Ed 201858

Distribution of results for

Programming

QM STEM Ed 201859

ANOVA

1. Select Analyze Compare Means One-Way ANOVA…

2. Select the variable programming and move into the Dependent List box.

3. Select campus and enter into the Factor box.

4. Click on Options… to open the Options box.

5. Click on Descriptive and Homogeneity of variance test

6. Select Continue and then OK or Paste.

QM STEM Ed 201860

ANOVA – descriptives

Descriptives

Programming

39 71.0256 17.99048 2.88078 65.1938 76.8575 30.00 98.00

13 62.8462 19.35565 5.36829 51.1497 74.5427 23.00 94.00

18 60.6667 11.84706 2.79238 54.7753 66.5581 33.00 75.00

8 46.0000 22.18751 7.84447 27.4508 64.5492 25.00 87.00

41 78.3659 23.03666 3.59772 71.0946 85.6371 16.00 100.00

119 69.4118 21.30585 1.95310 65.5441 73.2794 16.00 100.00

Inner city

Southern metropolitan

Eastern metropolitan

Country

Outer city

Total

N Mean Std. Dev iation Std. Error Lower Bound Upper Bound

95% Conf idence Interv al for

Mean

Minimum Maximum

Note that this table will only

appear if you request

Descriptive statistics in the

Options box.

QM STEM Ed 201861

ANOVA – checking homogeneity

Test of Homogeneity of Variances

Programming

1.995 4 114 .100

Levene

Stat istic df1 df2 Sig.

The p-value greater than 0.05

indicates no difference in the

population variances for each group.

QM STEM Ed 201862

ANOVA – test output

ANOVA

Programming

9710.645 4 2427.661 6.311 .000

43854.179 114 384.686

53564.824 118

Between Groups

Within Groups

Total

Sum of

Squares df Mean Square F Sig.

The p-values of the F test is less than O.05. We

therefore reject the null hypothesis that there is

no difference in the results for the programming

course across the five campuses.

The degrees of freedom of values

for each source of variance.

QM STEM Ed 201863

Means plot

The means plot helps

show the structure of

the data and where the

difference may occur.

QM STEM Ed 201864

ANOVA results


To investigate whether there was any difference in the

performance of students in the programming unit

across the five campuses, a one-way ANOVA was

performed.

This showed that the programming results differed significantly across the five campuses (F (4,114) = 6.31, p < 0.05).

However, where did these differences occur?

QM STEM Ed 201865

Post Hoc analysis

The ANOVA indicates that there was a significant difference in results across the campus – but it does not show which campuses were different – many comparisons were made.

Two approaches may be used:

Planned comparison where each comparison is investigated to assess a particular hypothesis.

Post hoc analysis involves finding where the differences occurred – all differences are investigated.

Post hoc tests help guard against Type 1 error.

QM STEM Ed 201866

Post Hoc analysis

Used for ANOVAs where the factors have

more than 2 levels – otherwise main effect is

sufficient.

Used for between groups comparisons.

There are difference tests available. We will

use Tukey HSD and Scheffe.

QM STEM Ed 201867

Post Hoc analysis

Select Analyze Compare Means One-Way ANOVA…

Click on Post Hoc… to open the Post Hoc box.

Click on Scheffe and Tukey check boxes

QM STEM Ed 201868

Post Hoc analysis – Tukey HSD

Multiple Comparisons

Dependent Variable: Programming

Tukey HSD

8.17949 6.28132 .690 -9.2318 25.5908

10.35897 5.58884 .348 -5.1329 25.8508

25.02564* 7.61245 .012 3.9245 46.1268

-7.34021 4.38706 .455 -19.5008 4.8204

-8.17949 6.28132 .690 -25.5908 9.2318

2.17949 7.13881 .998 -17.6088 21.9677

16.84615 8.81345 .317 -7.5841 41.2764

-15.51970 6.24290 .101 -32.8245 1.7851

-10.35897 5.58884 .348 -25.8508 5.1329

-2.17949 7.13881 .998 -21.9677 17.6088

14.66667 8.33410 .402 -8.4348 37.7681

-17.69919* 5.54563 .015 -33.0712 -2.3271

-25.02564* 7.61245 .012 -46.1268 -3.9245

-16.84615 8.81345 .317 -41.2764 7.5841

-14.66667 8.33410 .402 -37.7681 8.4348

-32.36585* 7.58079 .000 -53.3792 -11.3525

7.34021 4.38706 .455 -4.8204 19.5008

15.51970 6.24290 .101 -1.7851 32.8245

17.69919* 5.54563 .015 2.3271 33.0712

32.36585* 7.58079 .000 11.3525 53.3792

(J) Campus



Country

Outer city

Inner city


Country

Outer city

Inner city


Country

Outer city

Inner city



Outer city

Inner city



Country

(I) Campus

Inner city



Country

Outer city

Mean

Dif f erence

(I-J) Std. Error Sig. Lower Bound Upper Bound

95% Conf idence Interv al

The mean dif f erence is signif icant at the .05 level.*.

SPSS prints out a

complete matrix –

you need to ignore

repetitions.

As our factor had

5 levels there are

10 possible

comparisons.

QM STEM Ed 201869

Post Hoc analysis

Programming

8 46.0000

18 60.6667 60.6667

13 62.8462 62.8462

39 71.0256

41 78.3659

.110 .082

8 46.0000

18 60.6667 60.6667

13 62.8462 62.8462

39 71.0256

41 78.3659

.207 .165

Campus

Country



Inner city

Outer city

Sig.

Country



Inner city

Outer city

Sig.

Tukey HSDa,b

Schef fea,b

N 1 2

Subset f or alpha = .05

Means for groups in homogeneous subsets are display ed.

Uses Harmonic Mean Sample Size = 16.260.a.

The group sizes are unequal. The harmonic mean of the group sizes

is used. Type I error levels are not guaranteed.

b.

These show the

homogenous subsets.

Note that Scheffe and

Tukey HSD tests

produce the same

subsets but the p-

values are different.

QM STEM Ed 201870

One-way ANOVA with post hoc

analysis results


To investigate whether there was any difference in the

performance of students in the programming unit

across the five campuses, a one-way ANOVA was

performed.

This showed that the programming results differed significantly across the five campuses (F (4,114) = 6.31, p < 0.05).

Post Hoc analysis using Tukey’s HSD showed that there were two subsets of campuses whose means were not significantly different.

QM STEM Ed 201871

Two-way ANOVAs

Two-way ANOVAs have two factors (independent

variables). This requires:

Null hypothesis for each factor – main effects.

Also a possibility of interaction between the two

factors – interaction effect.

A significant interaction effect means that the

response at a particular level of one factor depends on

the level of the other factor.

In this situation, the main effects are ambiguous.

QM STEM Ed 201872

Two-way ANOVAs

For example, in a study of student exam performance in an introductory programming course you may want to investigate the effect of teaching approach (objects-first, objects-later) and learning approaches (deep, surface , achieving) on final grades.

To investigate this, a two-way between groups ANOVA may be used to compare:

exam performance between objects-first and objects-later students.

exam performance between deep, surface , achieving students.

the interaction between teaching approach and learning approach.

This is called a 2 x 3 ANOVA, That is, two independent variables – one with 2 levels and the other with 3 levels.

(In two-way ANOVAs one of the groups can be a repeated measure.)

QM STEM Ed 201873

Two-way ANOVAs

Consider the following result of the ANOVA:

exam performance is significantly influenced by teaching approach – main effect.

exam performance is significantly influenced by learning approach – main effect.

the interaction between teaching approach and learning approach is significant – interaction effect.

Therefore some combinations of teaching approaches and learning approaches may be more effective than others. In fact, teaching approach may not influence performance for all learning approaches – or vice versa.

n-way ANOVA

In an n-way ANOVA there are 2 or more factors each

containing two or more levels. An n-way ANOVA allows

us to analyse the main effects of all factors

simultaneously as well as any interactions between

them.

If two factors interact then the response at a particular level

of one factor depends on the level of one other factor.

If three factors interact then the response at a particular

level of one factor depends on the level of two other

factors. … and so on.

It is possible to have n-way ANOVAs; however, more than

3 or 4 is not common in education research.QM STEM Ed 2018

74

QM STEM Ed 201875

topic 5: parametric analyses - uppsala university

Documents