One Tailed and Two Tailed tests

One tailed tests: Based on a uni-directional hypothesisExample: Effect of training on problems using PowerPoint

Population figures for usability of PP are knownHypothesis: Training will decrease number of problems with PP

Two tailed tests: Based on a bi-directional hypothesisHypothesis: Training will change the number of problems

with PP

If we know the population meanIf we know the population mean

Mean Usability Index

7.257.00

6.756.50

6.256.00

5.755.50

5.255.00

4.754.50

4.254.00

3.75

Sampling Distribution

Population for usability of Powerpoint

Freq

uenc

y

1400

1200

1000

800

600

400

200

0

Std. Dev = .45 Mean = 5.65N = 10000.00

Unidirectional hypothesis: .05 level

Bidirectional hypothesis: .05 level

Identify region

• What does it mean if our significance level What does it mean if our significance level is .05?is .05? For a uni-directional hypothesisFor a uni-directional hypothesis For a bi-directional hypothesisFor a bi-directional hypothesis

PowerPoint example: PowerPoint example: • UnidirectionalUnidirectional

If we set significance level at .05 level, If we set significance level at .05 level, • 5% of the time we will higher mean by chance5% of the time we will higher mean by chance• 95% of the time the higher mean mean will be real95% of the time the higher mean mean will be real

• BidirectionalBidirectional If we set significance level at .05 levelIf we set significance level at .05 level

• 2.5 % of the time we will find higher mean by chance2.5 % of the time we will find higher mean by chance• 2.5% of the time we will find lower mean by chance2.5% of the time we will find lower mean by chance• 95% of time difference will be real95% of time difference will be real

Changing significance Changing significance levelslevels

•What happens if we decrease our What happens if we decrease our significance level from .01 to .05significance level from .01 to .05

Probability of finding differences that don’t Probability of finding differences that don’t exist goes up (criteria becomes more lenient)exist goes up (criteria becomes more lenient)

•What happens if we increase our What happens if we increase our significance from .01 to .001significance from .01 to .001

Probability of not finding differences that exist Probability of not finding differences that exist goes up (criteria becomes more conservative)goes up (criteria becomes more conservative)

• PowerPoint example:PowerPoint example: If we set If we set significance level at .05 level,significance level at .05 level,

• 5% of the time we will find a difference by chance5% of the time we will find a difference by chance• 95% of the time the difference will be real95% of the time the difference will be real

If we set If we set significance level at .01 levelsignificance level at .01 level• 1% of the time we will find a difference by chance1% of the time we will find a difference by chance• 99% of time difference will be real99% of time difference will be real

• For usability, if you are set out to find For usability, if you are set out to find problems: setting lenient criteria might problems: setting lenient criteria might work better (you will identify more work better (you will identify more problems)problems)

• Effect of decreasing significance level from Effect of decreasing significance level from .01 to .05.01 to .05 Probability of finding differences that don’t exist Probability of finding differences that don’t exist

goes up (criteria becomes more lenient)goes up (criteria becomes more lenient) Also called Also called Type I error (Alpha)Type I error (Alpha)

• Effect of increasing significance from .01 Effect of increasing significance from .01 to .001to .001 Probability of not finding differences that exist Probability of not finding differences that exist

goes up (criteria becomes more conservative)goes up (criteria becomes more conservative) Also called Also called Type II error (Beta)Type II error (Beta)

Degree of FreedomDegree of Freedom• The number of independent pieces of information The number of independent pieces of information

remaining after estimating one or more parametersremaining after estimating one or more parameters

• Example: List= 1, 2, 3, 4 Average= 2.5Example: List= 1, 2, 3, 4 Average= 2.5

• For average to remain the same three of the For average to remain the same three of the numbers can be anything you want, fourth is fixednumbers can be anything you want, fourth is fixed

• New List = 1, 5, 2.5, __ Average = 2.5New List = 1, 5, 2.5, __ Average = 2.5

Major PointsMajor Points• T tests: are differences significant?T tests: are differences significant?• One sample t tests, comparing one mean One sample t tests, comparing one mean

to populationto population• Within subjects test: Comparing mean in Within subjects test: Comparing mean in

condition 1 to mean in condition 2condition 1 to mean in condition 2• Between Subjects test: Comparing mean Between Subjects test: Comparing mean

in condition 1 to mean in condition 2in condition 1 to mean in condition 2

Effect of training on Powerpoint Effect of training on Powerpoint useuse

• Does training lead to lesser problems Does training lead to lesser problems with PP?with PP?

• 9 subjects were trained on the use of 9 subjects were trained on the use of PP.PP.

• Then designed a presentation with PP.Then designed a presentation with PP. No of problems they had was DVNo of problems they had was DV

Powerpoint study dataPowerpoint study data

• Mean = 23.89Mean = 23.89• SD = 4.20SD = 4.20

212421263227212518

Mean 23.89SD 4.20

Results of Powerpoint Results of Powerpoint study.study.

• ResultsResults Mean number of problems = 23.89Mean number of problems = 23.89

• Assume we know that without training the Assume we know that without training the mean would be 30, but not the standard mean would be 30, but not the standard deviation deviation Population mean = 30Population mean = 30

• Is 23.89 enough larger than 30 to conclude that Is 23.89 enough larger than 30 to conclude that video affected results?video affected results?

Sampling Distribution of Sampling Distribution of the Meanthe Mean

• We need to know what kinds of We need to know what kinds of sample means to expect if training sample means to expect if training has no effect.has no effect. i. e. What kinds of means if i. e. What kinds of means if = 23.89 = 23.89

This is the sampling distribution of the This is the sampling distribution of the mean.mean.

Sampling Distribution of Sampling Distribution of the Mean--cont.the Mean--cont.

• The sampling distribution of the The sampling distribution of the mean depends onmean depends on Mean of sampled populationMean of sampled population St. dev. of sampled populationSt. dev. of sampled population Size of sampleSize of sample

Mean Number of problems

7.257.00

6.756.50

6.256.00

5.755.50

5.255.00

4.754.50

4.254.00

3.75

Sampling Distribution

Number of problems with Powerpoint UseFr

eque

ncy

1400

1200

1000

800

600

400

200

0

Std. Dev = .45

Mean = 5.65

N = 10000.00

Cont.

Sampling Distribution of Sampling Distribution of the mean--cont.the mean--cont.

• Shape of the sampled populationShape of the sampled population Approaches normalApproaches normal Rate of approach depends on sample Rate of approach depends on sample

sizesize Also depends on the shape of the Also depends on the shape of the

population distributionpopulation distribution

Implications of the Central Implications of the Central Limit TheoremLimit Theorem

• Given a population with mean = Given a population with mean = and and standard deviation = standard deviation = , the sampling , the sampling distribution of the mean (the distribution distribution of the mean (the distribution of sample means) has a mean = of sample means) has a mean = , and , and a standard deviation = a standard deviation = / /nn. .

• The distribution approaches normal as The distribution approaches normal as nn, the sample size, increases., the sample size, increases.

DemonstrationDemonstration• Let population be very skewedLet population be very skewed• Draw samples of 3 and calculate meansDraw samples of 3 and calculate means• Draw samples of 10 and calculate meansDraw samples of 10 and calculate means• Plot meansPlot means• Note changes in means, standard Note changes in means, standard

deviations, and shapesdeviations, and shapes

Cont.

X

20.018.0

16.014.0

12.010.0

8.06.04.02.00.0

Skewed Population F

requ

ency

3000

2000

1000

0

Std. Dev = 2.43

Mean = 3.0

N = 10000.00

Parent PopulationParent Population

Cont.

Sampling Distribution Sampling Distribution nn = = 33

Sample Mean

13.0012.00

11.0010.00

9.008.00

7.006.00

5.004.00

3.002.00

1.000.00

Sampling Distribution

Sample size = n = 3Fr

eque

ncy

2000

1000

0

Std. Dev = 1.40

Mean = 2.99

N = 10000.00

Cont.

Sampling Distribution Sampling Distribution nn = = 1010

Sample Mean

6.506.00

5.505.00

4.504.00

3.503.00

2.502.00

1.501.00

Sampling Distribution

Sample size = n = 10F

requ

ency

1600

1400

1200

1000

800

600

400

200

0

Std. Dev = .77

Mean = 2.99

N = 10000.00

Cont.

Demonstration--cont.Demonstration--cont.• Means have stayed at 3.00 throughout--Means have stayed at 3.00 throughout--

except for minor sampling errorexcept for minor sampling error• Standard deviations have decreased Standard deviations have decreased

appropriatelyappropriately• Shapes have become more normal--see Shapes have become more normal--see

superimposed normal distribution for superimposed normal distribution for referencereference

One sample t test cont.One sample t test cont.• Assume mean of population known, but Assume mean of population known, but

standard deviation (SD) not knownstandard deviation (SD) not known• Substitute sample SD for population SD Substitute sample SD for population SD

(standard error)(standard error)• Gives you the t statisticsGives you the t statistics• Compare Compare t t to tabled values which show to tabled values which show

critical values of tcritical values of t

tt Test for One Mean Test for One Mean• Get mean difference between Get mean difference between

sample and population mean sample and population mean • Use sample SD as variance metric = Use sample SD as variance metric =

4.404.4048.1

46.111.6

940.4

89.2330

ns

Xt

Degrees of FreedomDegrees of Freedom• Skewness of sampling distribution of Skewness of sampling distribution of

variance decreases as variance decreases as nn increases increases• tt will differ from will differ from zz less as sample size less as sample size

increasesincreases• Therefore need to adjust Therefore need to adjust tt accordingly accordingly• dfdf = = nn - 1 - 1• tt based on based on dfdf

Looking up critical t (Table Looking up critical t (Table E.6)E.6)

Two-Tailed Significance Level df .10 .05 .02 .01 4 1.812 2.228 2.764 3.169

5 1.753 2.131 2.602 2.947 6 1.725 2.086 2.528 2.845 7 1.708 2.060 2.485 2.787 8 1.697 2.042 2.457 2.750 9 1.660 1.984 2.364 2.626

ConclusionsConclusions• Critical t= Critical t= nn = 9, = 9, tt.05.05 = 2.62 (two tail = 2.62 (two tail

significance)significance)• If If tt > 2.62, reject > 2.62, reject HH00

• Conclude that training leads to less Conclude that training leads to less problemsproblems

Factors Affecting Factors Affecting tt• Difference between sample and Difference between sample and

population meanspopulation means• Magnitude of sample varianceMagnitude of sample variance• Sample sizeSample size

Factors Affecting DecisionFactors Affecting Decision• Significance level Significance level • One-tailed versus two-tailed testOne-tailed versus two-tailed test