assignment 2- slides
TRANSCRIPT
![Page 1: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/1.jpg)
P S Y C H O L O G Y 3 8 0 0 , L A B 0 0 2
![Page 2: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/2.jpg)
In Today’s Lab…
! refresher on populations, hypotheses, error
! Monte Carlo methods and the t-distribution
! exploring assumption violations using Monte Carlo
! this week’s assignment (assignment #2)
![Page 3: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/3.jpg)
![Page 4: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/4.jpg)
Populations versus Samples
population: includes all people or items with the characteristic one wishes to understand
sample: a subset of the desired population, which is (ideally) representative of the population and obtained through random sampling
In statistics, we test samples, and use the results from these samples to make inferences about the populations from which they were drawn.
![Page 5: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/5.jpg)
Null Hypothesis
Alternative Hypothesis (Claim Being Tested)
• null hypothesis suggests that there is no difference between the two populations of interest, or that the effect is in the opposite direction than expected
• example: study looking at average self-reported hunger of males versus females after watching the Food Network (H0: µfemales = µmales)
• reflects the hypothesized effect (opposite of null hypothesis)
• example: males and females report significantly different levels of hunger (HA: µfemales ! µmales)
Hypotheses in the t-Test
Note that hypotheses are stated in terms of population parameters (µ), but are tested using sample statistics (e.g., )
!
x
![Page 6: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/6.jpg)
The results of our analysis will tell us whether or not we reject our null hypothesis using the significance (p-value; probability value).
If p < .05 (including p < .01, p < .001) ! reject H0 (go with the effect specified in HA) ! conclude that significant effect exists
If p > .05 (i.e., ns) ! do not reject H0 (go with the effect specified in H0) ! conclude that no significant effect exists
Hypothesis Testing in the t-Test
![Page 7: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/7.jpg)
Hypothesis Testing in the t-Test
Mistakes happen…
![Page 8: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/8.jpg)
Type I error ! reject the null hypothesis (H0) when it is in fact true ! conclude populations differ when they are essentially the same
Type II error ! do not reject the null hypothesis (H0) when it is actually false ! conclude population do not differ when significant differences exist
Both types of errors are bad, and you don’t want to make either!
Types of Errors
![Page 9: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/9.jpg)
![Page 10: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/10.jpg)
t-distribution
F-distribution
!2-distribution
sampling distributions of various statistics…
Sampling Distributions
![Page 11: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/11.jpg)
In your t-test outputs from last week, you were seeing and reporting t-statistics (i.e., the t-obtained value for a given t-test):
Each individual t-statistic represents a measure of the difference between two means, given the variance and size of the samples:
!
t =xa " xbs2
na+s2
nb
Sampling Distributions: t-Distribution
![Page 12: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/12.jpg)
! a distribution is derived by taking an infinite number of samples from any given pairs of populations with specified characteristics and calculating t-values for the pairs of samples
! plotting t-values gives us the t-distribution
t-distribution
Sampling Distributions: t-Distribution
![Page 13: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/13.jpg)
Population 1 Population 2
Take a given sample of scores from both populations (e.g., n = 5).
Calculate a t-statistic for that comparison using the standard formula (e.g., t = 3.46).
Sampling Distributions: t-Distribution
![Page 14: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/14.jpg)
These might be the samples for pair 2
(t = -2.97)
Population 1 Population 2
Sampling Distributions: t-Distribution
![Page 15: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/15.jpg)
And these might be the samples for the pair 3 t-statistic calculation
(t = 0.21)
…And it would go on from here to generate a number of t-values that form a t-distribution…
Population 1 Population 2
Sampling Distributions: t-Distribution
![Page 16: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/16.jpg)
When populations are the same, this should result in t-obtained values near 0 on our null hypothesis (H0) distribution
! smaller values indicate less dramatic differences between our two means ! with small t-values, we would not reject H0 (conclude that our means do not differ significantly)
Sampling Distributions: t-Distribution
![Page 17: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/17.jpg)
95%
If two population means are equal, then a t-statistic comparing sample means will fall between these two dotted lines (the non-rejection region) 95% of the time (at " = .05 in a two-tailed test)
! 5% chance of concluding that means differ when populations are, in fact, essentially the same (Type I error)
2.5% 2.5%
Sampling Distributions: t-Distribution
adds to 100% (accounting for all possible outcomes)
![Page 18: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/18.jpg)
99%
If two population means are equal, then a t-statistic comparing sample means will fall between these two dotted lines (the non-rejection region) 99% of the time (at " = .01 in a two-tailed test)
! 1% chance of concluding that means differ when populations are, in fact, essentially the same (Type I error)
0.5% 0.5%
Sampling Distributions: t-Distribution
Researcher sets the level of alpha based on Type I error tolerance.
![Page 19: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/19.jpg)
Sampling Distributions: t-Distribution
When populations are different, this should result in larger t-obtained values that fall at the extremes (positive or negative) of our our null hypothesis (H0) distribution
! larger values indicate more dramatic differences between our two means ! with large t-values, we would reject H0 (conclude that our means differ significantly)
![Page 20: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/20.jpg)
Sampling Distributions: t-Distribution
If our population means do indeed differ, then rejecting our null hypothesis after obtaining large t-values is the correct decision (power).
If our population means do not differ, then rejecting our null hypothesis after obtaining large t-values is the incorrect decision (Type I error).
In practical research settings, we do not know whether the population means truly differ or not… As a result, we can never know whether we have made the right conclusion or whether we have committed an error.
! importance of good research design and addressing the assumptions of a test properly
![Page 21: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/21.jpg)
H0 is true H0 is false
do not reject H0 (conclude H0 is true)
Type II error (#)
reject H0 (conclude HA is true)
Type I error (") w
hat w
e co
nclu
de
(kno
wn
afte
r we
carr
y ou
t tes
t)
what is true (unknown to us)
Types of Errors
We cannot commit both types of errors in the same analysis, because they hinge on two different “realities”.
![Page 22: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/22.jpg)
Relationship exists between Type I and Type II error: if we decrease the probability of one, the probability of the other increases
• with very small alpha values ! low Type I error rate ! high Type II error rate: becomes very difficult to obtain t-values large enough to reject H0 (could erroneously conclude that H0 is true when in reality it is not)
Relating Type I and Type II Error
balancing act between the two errors
![Page 23: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/23.jpg)
The t-Distribution and Monte Carlo
Generating such a distribution on our own would be overwhelming!
• we can use the Monte Carlo statistical package to investigate these sampling distributions
! does not use SPSS ! involves running computer simulations of statistical tests (e.g., t-tests) using artificial data sets and huge numbers of repetitions
![Page 24: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/24.jpg)
The t-Distribution and Monte Carlo
• if we can generate distributions using Monte Carlo, we can also use this programs to examine what happens to these distributions when certain assumptions underlying tests are violated
Assumptions Underlying the t-Distribution (p. 58 in textbook)
(1) independent random sampling
(2) normality
(3) homogeneity of variance
violating these assumptions can affect Type I and Type II error rates
![Page 25: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/25.jpg)
![Page 26: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/26.jpg)
N(0,1)5 vs. N(0,1)5
shape of population (N = normal)
mean (µ)
variance ("2)
sample size (n)
1) define our two populations
2) MONTE obtains random samples of specified sample size (via simulations, up to 1000)
3) MONTE computes and outputs statistics pertaining to simulations
4) we compare obtained t-distribution values to known t-distribution values to assess how our specified parameters affected our results
Monte Carlo: Overview of Steps
![Page 27: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/27.jpg)
Accessing MONTE on SSC Network
Start ! My Computer
![Page 28: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/28.jpg)
Accessing MONTE on SSC Network
double-click to select your “L” drive (under “Network Drives”)
![Page 29: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/29.jpg)
Accessing MONTE on SSC Network
double-click to select the “Course Library” folder
double-click to select the “Psychology” folder
![Page 30: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/30.jpg)
Accessing MONTE on SSC Network
double-click to select the “MONTE.exe” program
specify that you want to “Run” the program
![Page 31: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/31.jpg)
Accessing MONTE on SSC Network
![Page 32: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/32.jpg)
MONTE Example 1
" What is the sample size for 1? 5
" What is the sample size for 2? 5 " What is the mean for population 1? 0
" What is the mean for population 2? 0 " What is the standard dev. for population 1? 1 (!1=1) " What is the standard dev. for population 2? 1 (!1=1) " What is the shape of population 1 (N, E, or R)? N
" What is the shape of population 2 (N, E, or R)? N " What is the value of t required for significance for at the .05 level? 2.306 (from tables) " What is the value of t required for significance for at the .01 level? 3.355 (from tables) " Number of samplings? 1000 " Do you only want the summary output? Y " Do you want the frequency distribution for the populations? N
N(0,1)5 vs. N(0,1)5 …no assumptions violated
![Page 33: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/33.jpg)
Obtaining the Results
MONTE Example 1
…navigate to your U: drive
![Page 34: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/34.jpg)
MONTE Example 1
results from all simulations will be stored in one file (double-click to open)
click “Open” to run the file
Obtaining the Results
![Page 35: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/35.jpg)
MONTE Example 1
Obtaining the Results
![Page 36: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/36.jpg)
MONTE Example 1
Note: given that the results are derived from simulation of random samples, all output will vary (but general trends will remain)
![Page 37: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/37.jpg)
MONTE Example 1
Note: given that the results are derived from simulation of random samples, all output will vary (but general trends will remain)
this is just a repeat of the info that you provided to MONTE (lets you know which results are which in the output document)
![Page 38: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/38.jpg)
MONTE Example 1
Note: given that the results are derived from simulation of random samples, all output will vary (but general trends will remain)
Results for the first 3 simulations (of 1000):
! for each: t-obtained value, mean of each population, variance of each population (two populations being compared)
! these values represent sample values
! if running multiple simulations, can look to see how these vary
![Page 39: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/39.jpg)
MONTE Example 1
Note: given that the results are derived from simulation of random samples, all output will vary (but general trends will remain)
At the .05 level: 4.4% significant t-values
At the .01 level: 0.9% significant t-values !
positive t's + negative t'snumber of simulations
=24 + 201000
= .044
!
positive t's + negative t'snumber of simulations
=2 + 71000
= .009
![Page 40: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/40.jpg)
MONTE Example 1
! As expected, with no violations of our assumptions… " just under 5% of t-values are significant when # = .05 " just under 1% of t-values are significant when # = .01 " these significant t-values are Type I errors (because, in fact, our two
population means as provided to MONTE were equal)
! What we might want to know is: “If we violate one of our assumptions, will this Type I error rate go up?”
![Page 41: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/41.jpg)
MONTE Example 2
" What is the sample size for 1? 10
" What is the sample size for 2? 10 " What is the mean for population 1? 1
" What is the mean for population 2? 1 " What is the standard dev. for population 1? 1 (!1=1) " What is the standard dev. for population 2? 3 (!9=3) " What is the shape of population 1 (N, E, or R)? N
" What is the shape of population 2 (N, E, or R)? N " What is the value of t required for significance for at the .05 level? 2.101 (from tables) " What is the value of t required for significance for at the .01 level? 2.878 (from tables) " Number of samplings? 1000 " Do you only want the summary output? Y " Do you want the frequency distribution for the populations? N
N(1,1)10 vs. N(1, 9)10 …equal variances assumption violated
![Page 42: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/42.jpg)
MONTE Example 2
At the .05 level: 5.6% significant t-values
At the .01 level: 1.4% significant t-values !
positive t's + negative t'snumber of simulations
=25 + 311000
= .056
!
positive t's + negative t'snumber of simulations
=7 + 71000
= .014
![Page 43: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/43.jpg)
MONTE Example 2
! recall: we expect about 5% of the t-values to be significant at ! = .05, and 1% at ! = .01
! by violating the assumption of homogeneity of variance, we found slightly more significant t-values than expected at ! = .05 (5.6%), and at ! = .01 (1.4%)
! the significant t-values are still considered to be error here because our two true means are, in fact, equal
What do the results look like when the population means differ?
![Page 44: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/44.jpg)
MONTE Example 3
" What is the sample size for 1? 10
" What is the sample size for 2? 10 " What is the mean for population 1? 0
" What is the mean for population 2? 2 " What is the standard dev. for population 1? 1 (!1=1) " What is the standard dev. for population 2? 1 (!1=1) " What is the shape of population 1 (N, E, or R)? N
" What is the shape of population 2 (N, E, or R)? N " What is the value of t required for significance for at the .05 level? 2.101 (from tables) " What is the value of t required for significance for at the .01 level? 2.878 (from tables) " Number of samplings? 1000 " Do you only want the summary output? Y " Do you want the frequency distribution for the populations? N
N(0,1)10 vs. N(2,1)10 …means actually differ in the populations
![Page 45: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/45.jpg)
MONTE Example 3
At the .05 level: 99.0% significant t-values
At the .01 level: 94.1% significant t-values !
positive t's +negative t'snumber of simulations
=990 + 01000
= .990
!
positive t's +negative t'snumber of simulations
=941+ 01000
= .941
![Page 46: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/46.jpg)
MONTE Example 3
• MONTE program tells us that with our particular populations and samples, 99 of 1000 t-obtained values were significant
! that is, 99% of the time, we will catch the significant difference between our two populations at the .05 level ! this value is no longer error… it is “power”
power: the probability that if a significant difference exists in real life between the populations, our statistical test will show that difference
Can we also obtain an understanding of error from this information?
![Page 47: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/47.jpg)
MONTE Example 3 (Power/Beta)
1 – POWER = Type II Error Rate
! also: 1 – Type II Error Rate = POWER ! also: POWER + Type II Error Rate = 1
Type II error (" or beta) do not reject the null hypothesis (H0) when it is actually false (i.e., you conclude no differences between means when in fact a significant difference exists)
• if MONTE tells us that 99% of the time, we will catch the significant difference between our two populations at the .05 level
! 1% of the time we will conclude that there the populations mean do not differ despite there being an effect of our manipulation (commit Type II error)
![Page 48: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/48.jpg)
![Page 49: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/49.jpg)
Assignment Format
! not to be written like an APA-style results section as we are not running statistical tests to assess certain hypotheses (but adhere to APA formatting)
! instead: more like a position paper, where you support your conclusion with evidence
! because MONTE takes random samples, everyone will have different results (don’t panic)
![Page 50: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/50.jpg)
! define Type I error and power
! describe the following trends, providing concrete examples from your output, and explaining why they occur: " effect of sample size on power " effects of alpha level on power " effects of sample size on Type I error " effects of alpha level on Type I error
! keep the discussion to two, double-spaced pages
! summarize your results in an APA-style table (extra page)
! submit your MONTE output (neat, labeled, headings where needed, can delete frequency info)
What to Include…
![Page 51: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/51.jpg)
What to Include…
Note: This is an example figure. It is not specific to your assignment question and should not be copied.
Simulation Distribution Observed % error at 5% level 1% level
1 N(1,1)12 – N(0,1)12 5.3 .90 2 N(0,1)25 – N(0,1)25 4.0 .80 3 N(1,1)5 – N(0,1)15 4.0 .60
![Page 52: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/52.jpg)
table title (italicized, proper
capitalization)
table number (no italics or
period)
only horizontal lines
Note, if necessary
double-spaced if possible
Tables: APA Formatting
* when in doubt, check out APA sources
only first word capitalized in table
headings
![Page 53: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/53.jpg)
Next Week
! single-factor analysis of variance (a.k.a. one-way ANOVA)
! hand in Monte Carlo assignment at the beginning of lab
! I will return marked t-test assignments
! for those who want extra practice: check out the weekly practice assignments posted on Sakai (Practice Labs folder)
![Page 54: Assignment 2- Slides](https://reader035.vdocument.in/reader035/viewer/2022062702/554a5849b4c9054b328b4d0d/html5/thumbnails/54.jpg)
Simulations to Run in MONTE
Simulations Critical values
" = .05 " = .01
N(1, 2)5; N(0, 2)5 2.306 3.355
N(1, 2)10; N(0, 2)10 2.101 2.878
N(1, 2)20; N(0, 2)20 2.025 2.713
N(1, 2)40; N(0, 2)40 1.990 2.641
N(1, 2)100; N(0, 2)100 1.980 2.617
N(1, 2)5; N(1, 2)5 2.306 3.355
N(1, 2)10; N(1, 2)10 2.101 2.878
N(1, 2)20; N(1, 2)20 2.025 2.713
N(1, 2)40; N(1, 2)40 1.990 2.641
N(1, 2)100; N(1, 2)100 1.980 2.617
power
Type I error
Important: the second value in the parentheses represents the standard deviation ($) rather than the variance ($2) … can be entered into MONTE as is