chapter two research - ewyner.com 2/psychology slides 2.… · (cheung & rensvold, 2002). the...

34
Chapter Two Research Statistics Designs Ethics

Upload: others

Post on 16-Apr-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Chapter Two

Research

StatisticsDesignsEthics

Page 2: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Part One: StatisticsResearchers collect data to test for predicted relationships between independent and dependent variables, display these relationships in written and graphical form, and interpret them.• Festinger&Carlsmith(1959)• basic science terminology

samples and populations – sample statistics

• variables• two-variable bar graphs

good and bad graphs – a real bad graph

• p valuespowerexamples of statistical significancetype I and type II errors

Page 3: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Festinger 5 & Carlsmith (1959)Leon Festinger’s cognitive dissonance theory holds that people tend to change their attitudes to match their actions if they don’t have a compelling reason not to. To test this, he had 60 male Stanford undergraduates do two 30-minute monotonous tasks: putting spools onto and off of a peg board, and turning square pegs a quarter-turn at a time. Afterward, some of them were hired as research assistants, for either $1 or $20, to tell the next person that the experiment was fun and exciting. Finally, participants were interviewed with questions like “Were the tasks interesting and enjoyable?” on a scale of -5 to +5.

5.0

2.5

0.0

-2.5

-5.0 none lie, for $20 lie, for $1

task afterward

ratin

g of

inte

rest

and

enjo

ymen

t

Page 4: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Basic science terminologySocial sciences, such as psychology, are different from physical sciences, such as chem-istry, in significant aspects. Physical sciences involve laws of nature typically established through algebra or calculus. Social sciences involve theories of behavior typically estab-lished through statistical analysis. Physical sciences are based more on calculations and social sciences are based more on critical reasoning. Nonetheless, both are sciences, both use the scientific method, and the terms below apply to both.Term Definition ExampleScience systematic knowledge

gained by observation or experimentation

psychology

Theory unifying body of knowledge explaining em-pirically demonstrated facts

cognitive dissonance theory

Hypothesis testable prediction based on theory

Festinger & Carlsmith (1959)

Falsifiability ability to be tested boys are taller than girlsReplication performing similar

experiments to check for similar results

Aronson & Carlsmith (1963)

Page 5: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Samples and PopulationsData are collected from samples, but researchers want to know about entire populations.Term Definition ExamplePopulation the group being studied AmericansSample the subset of the

population from which data are actually collected

300 college undergraduates

Results from a study can only be generalized to populations represented by the sample.The college sophomore problem is that a large per-centage of social science stud-ies use college undergraduates to form their samples, but college undergraduates tend to be different than the overal population in some respects.

Page 6: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Sample StatisticsA statistic is a value calculated from a sample.Statistic Measures Range Zero representsSample Size (n) how many in sample n > 0 There is no sample.

Mean (x) average any (usually x > 0) There is none of what is being measured.

Standard Deviation (s)

variability s ≥ 0 All values are equal.

Proportion (p) percentage 0 ≤ p ≤ 1 No one in the sample meets the given criteria.

Correlation Coefficient (r)

how related one numerical variable is to another

-1 ≤ r ≤ 1 The two variables are completely unrelated.

^ ^

Page 7: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

VariablesScience studies the relationship between variables. The examples below are from Festinger and Carlsmith’s 1959 study on cognitive dissonance.Term Definition ExampleIndependent Variable variable whose values are

known (often chosen) before those of the dependent variable

amount of money paid

Level category of an indepenent variable

$1 or $20

Dependent Variable variable whose values are known (often measured) after those of the independent variable

how enjoyable the task was

Operational Definition definition of a variable such that it can be categorized or measured

self-rating of “interesting and enjoyable” from -5 to +5

Extraneous Variable variable that influences the dependent variable but is unrelated to the independent variable

need for excitement

Page 8: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Two-variable bar graphsA bar graph should be used to display the relationship between a categorical indepen-dent variable and a numerical dependent variable.Region DescriptionBars There is one bar for each level of the independent variable, and they are

not connected.x-axis Each bar is labeled. The independent variable is stated below.y-axis There is an appropriate scale. Unless there is a specific reason to do

otherwise, the scale should start at the lowest possible or realistic value of the dependent variable (typically zero) and go up to the highest possible or realistic value of the dependent variable. The dependent variable is stated to the side.

Title The title clarifies the relationship being studied, making the graph understandable without additional information.

Legend There should not be a legend for a graph with only one independent variable.

Page 9: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Comparing good and bad graphsMicrosoft Excel (and other data software) can be very helpful in creating graphs. How-ever, default settings will often result in bad graphs, such as the one on the left below.Problem with graph on left Consequenceno x-axis label; untitled categories not known what is being comparedno y-axis label or title not known what is being measuredy-axis doesn’t start at zero misleading comparisoninconsistent decimal places distracting, unprofessional“Series1” confusing because it is meaningless

81.35  

81.4  

81.45  

81.5  

81.55  

81.6  

81.65  

81.7  

81.75  

81.8  

81.85  

group  1   group  2  

Series1  

Effect of Noise on Recall100%

75%

50%

25%

0% quiet noisy

backgroundpr

opor

tion

of

wor

ds re

calle

d

Page 10: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

A real bad graphSometimes graphs are designed to be misleading, for political or other reasons. The TV graphic below, which aired on December 12, 2011, is an example. The graph on the right shows the same information displayed appropriately.Purposely misrepresenting data is a violation of APA ethical guidelines.

If Bush tax cuts expire40%

30%

20%

10%

0% 2011 2013

tax yearto

p ta

x rat

e

Page 11: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

P ValuesA p value is the probability of a predicted event happening by chance, that is, given there is no actual basis for the prediction. The main purpose of a statistics course is to learn how to calculate and interpret p values (although calculating p values is beyond the scope of this course). Some hypothetical predictions and results are shown below. Hypothesis Result Distribution p valueA certain coin lands on heads more than tails.

n = 25 coin flipsp = 15/25 are heads

binomial .097

People watch more than 2 hours of TV per day.

n = 30 peoplex = 2.6* hourss = 1.5 hours

normal .014

Graduation rate varies with job type.

p1 = 45/51 with no jobp2 = 80/86 in retailp3 = 38/48 in labor

chi-square .059

If p is less than .05, the result is considered statistically significant. In this case, the researchers conclude that their hypothesis was correct and mark the result with *. If p is less than .01, they are even more confident of their conclusion and mark the result with **. Likewise, results with p less than .001 are marked with ***.

^

^

^

^

Page 12: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

PowerInferential statistics involves the calculation and analysis of p values to conclude whether a finding represents an actual phenomenon or is merely conicidental. Since a p value is the probability of a predicted event happening given there is no actual basis for the prediction, lower p values represent greater likelihoods that there is in fact a ba-sis for the prediction, that is, that the sample results were not coincidental but instead represent the population as hypothesized.Lower p values result from tests with more power, which comes from three sources: large sample size, low variance, and large effect size. Consider the following data.Data Set Sample Size Standard Deviation Effect Sizex1 = 14, 28, 21x2 = 16, 24, 27

Only n = 3 people participated.

There is a lot of variation within each sample: s1 = 7.0, s2 = 5.7.

There is not much difference between the group means: x1 = 21.0, x2 = 22.3.

x1 = 16, 18, 15, 19, 20, 21, 19x2 = 21, 25, 22, 28, 21, 24, 23

More people (n = 7) participated.

The data within each sample are fairly close together: s1 = 2.1, s2 = 2.0.

There is a fairly large difference between the group means: x1 = 18.3, x2 = 23.4***.

Page 13: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

An example of statistically significant correlationsFrom a questionnaire given to 409 Canadian teachers, Klassen, Perry, and Frenzel (2012) (p. 155) calculated correlations between autonomy support, relatedness with colleagues, relatedness with students, work engagement, and emotional exhaustion.

between elementary and secondary teachers for the dependentvariables, with elementary teachers reporting higher relatednesswith students than secondary teachers (p � .01), but with modesteffect size (d � 0.31).

Table 2 presents the intercorrelation matrix for the study vari-ables, first for elementary and secondary teachers and then for thetotal sample, including middle school teachers. Direction andmagnitude of correlations were similar for elementary and second-ary teachers, and there were no significant differences betweencorrelations for the elementary and secondary teachers (usingFisher’s Z transformation test). All bivariate relationships were inthe expected direction (i.e., positive correlations between related-ness and work engagement and negative correlations betweenrelatedness and emotional exhaustion). Satisfaction of the need forrelatedness with students showed significantly stronger bivariaterelationships with work engagement than satisfaction of the needfor relatedness with colleagues for both groups of teachers and forthe total sample (all ps � .001). For emotional exhaustion, thebivariate relationships with the relatedness-with-students variablewere nominally, but not statistically significantly higher than thebivariate relationships with the relatedness-with-colleagues vari-able.

Confirmatory factor analysis and SEM. For the full sample(i.e., elementary, middle school, and secondary teachers combined;n � 409), the measurement model with autonomy support, col-league and student relatedness, work engagement, and burnoutshowed good fit (�2/df � 2.21, CFI � .961, TLI � .962,RMSEA � .054). The model with colleague relatedness andstudent relatedness as separate factors fit the data better than themodel with a single combined relatedness factor (p � .001). Inaddition, the model with work engagement and emotional exhaus-tion as separate factors fit the data better than a model with a singlecombined work motivation factor (p � .001), suggesting thatengagement and emotional exhaustion are separate factors. Eachindicator loaded significantly (p � .01) onto its associated latentfactor and ranged from � � .59 to � � .90 (mean � � .79).

Figure 2 shows the path diagram of the standardized parameterestimates for the structural model. The hypothesized structural

model showed an adequate fit for the overall sample (�2/df � 2.34,CFI � .961, TLI � .953, RMSEA � .06). Autonomy support wassignificantly associated with relatedness with colleagues and stu-dents. Relatedness with students was significantly associated withhigher levels of work engagement and lower levels of emotionalexhaustion (p � .01). Relatedness with colleagues was not signif-icantly associated with either work engagement or emotional ex-haustion. The variance accounted for by both outcomes in ourmodel was significant: work engagement, R2 � .65; and emotionalexhaustion, R2 � .07.

Next, we tested for multigroup invariance of the measurementmodel for the elementary (n � 184) and secondary teachers (n �149). In establishing baseline models, the basic factor structurewas maintained for all groups, with all parameters allowed to differacross groups. Hierarchical goodness of fit was evaluated bychanges in the CFI (�CFI), wherein a �CFI less than or equal to.01 indicates invariance when additional constraints are imposed(Cheung & Rensvold, 2002). The use of �CFI is superior to theuse of ��2 as a test of invariance because �CFI is not affected bysample size and is preferable to the use of other goodness-of-fitchange indices that possess undesirable qualities that influence theaccuracy of measurement when invariance constraints are added(see Cheung & Rensvold, 2002, for further discussion).

The measurement model fit the data for both groups, indicatinggood fit of the data to the model. For the elementary teachers, fitindices were acceptable (�2/df � 1.78, CFI � .947, RMSEA �.065). For the secondary teachers, fit indices indicated good fit(�2/df � 1.33, CFI � .973, RMSEA � .047). The unconstrainedmodel (consisting of the baseline models for the two school-levelgroups) showed a good fit (�2/df � 1.58, CFI � .956, RMSEA �.042), suggesting a common factor structure across the two groups.Constraining the factor loadings across the two groups resulted incontinued excellent fit (�CFI � .002), suggesting that factorweights were invariant across the two groups. Constraining struc-tural variances yielded nonsignificant changes in fit (�CFI ��.001), suggesting no difference in structural variances across thetwo groups. Constraining structural intercepts yielded nonsignifi-cant changes in fit (�CFI � .000). Finally, constraining structural

Table 2Intercorrelation Matrix for Variables in Study 1 for Elementary, Secondary, and Total Teachers

Variable 1 2 3 4 5

Elementary and secondary teachers1. Autonomy support — .33�� .32�� .42�� �.25��

2. Relatedness with colleagues .31�� — .34�� .36�� �.15�

3. Relatedness with students .32�� .32�� — .59�� �.24��

4. Work engagement .43�� .29�� .69�� — �.48��

5. Emotional exhaustion �.31�� �.10 �.19� �.48�� —

Total teachers1. Autonomy support — .32�� .32�� .40�� �.29��

2. Relatedness with colleagues — .33�� .31�� �.12�

3. Relatedness with students — .62�� �.23��

4. Work engagement — �.49��

5. Emotional exhaustion —

Note. In the first block, elementary teachers (n � 184) are above the diagonal and secondary teachers (n � 149)are below the diagonal. The second block displays coefficients for all teachers (n � 409) and includes teachersfrom middle schools.� p � .05. �� p � .01.

155TEACHERS’ RELATEDNESS WITH STUDENTS

Page 14: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Type I and Type II ErrorsAs described above, when a p value is below .05, the probability of the results turning out that extreme by chance is low enough that they were unlikely to be coincidental, and therefore researchers will conclude that their hypothesis was correct. However, since a p value is a probability, there is always the possibility that the results actually were a coincidence despite the low p value. Likewise, if p is not below .05, the results are not remarkable enough to rule out the possibility of a coincidence, and therefore researchers will not conclude that their hypothesis was correct. However, not being able to rule out a coincidence is much different from proving that the results were a coincidence. Nothing can be proved with statistics.Actual Situation p < .05 p > .05Hypothesis is correct The researchers correctly

conclude that their hypothesis was correct.

Type II error: The re-searchers’ hypothesis was correct, but they disregard it.

Hypothesis is incorrect Type I error: The re-searchers’ hypothesis was incorrect, but they keep it.

The researchers correctly conclude that their hypothesis was incorrect.

Page 15: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Part Two: Research DesignsResearchers need to carefully plan their methods of data collection and interpret their findings appropriately.• approaches to data collection• types of designs• correlation and causation

confounds – correlation does not imply causation – reasons for results – procedural artifacts – Waber, Shiv, Carmon, & Ariely (2008) – Rosenthal84 & Jacobson (1966) – sequence effects

uses of different designs – choosing a design

• factorial designs• post hoc analysis

Page 16: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Approaches to data collectionThe approaches below, some of which overlap, are common in social science research.Approach DescriptionLaboratory Research Data are collected under controlled conditions.Field Research Data are collected in real-world settings.Naturalistic Observation Natural behaviors are observed and not influenced.Survey Data are self-reported.Meta-analysis Statistical methods are used to combine relevant past

studies into overall conclusions.Case Study Extensive qualitative data from a single individual or

unit is collected over time.Archival Research Past data are gathered from existing sources.

Page 17: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Types of designsThe design of an experiment is the plan for the collection, analysis, and interpretation of data.True experiments use random assignment: Each participant takes part in a single condition (between-participants) that is randomly assigned, or takes part in each condition (within-participants) in random order or simultaneously.Quasi-experiments are the same as experiments except without random assignment.Correlational studies are the same as quasi-experiments except that the independent variable is numerical rather than having different conditions.

Page 18: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Correlation and CausationA correlational relationship is one in which values of the independent variable can predict corresponding values of the dependent variable.A causal relationship is one in which the independent variable affects the dependent variable.A causal relationship can be used to make predictions. For example, a child’s age affects his reading speed, so age can be used to predict reading speed.However, a correlational relationship cannot be used to establish causa-tion. For example, a child’s height can be used to pre-dict reading ability (be-cause children get taller as they get older), but it would be an error to con-clude that height affects reading speed.

Page 19: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

ConfoundsA confounding variable is one that affects the dependent variable and is linked to, but not caused by, the independent variable. Extraneous variables also affect the dependent variable, but they do so randomly rather than being linked to the independent variable. Confounding variables can greatly reduce the value of a study, because the results can-not be interpreted causally: The results may indicate what relationship exists between the independent and dependent variable, but not why it exists. The relationship could be due to the independent variable’s effect on the dependent variable as hypothesized, but it could also be due to confounding variables like in the examples below.Hypothesis Extraneous Variables Confounding VariablesGeneral Case: A affects Z. B and C that affect Z but

aren’t related to AD that affects A and Z simultaneously, or E that affects Z and is correlated with, but not affected by, A

Doing all reading assignments increases test scores.

writing skills, intelligence, alertness during test

reading comprehension skills, self-efficacy, motivation

Blacks are genetically more predisposed to violence than Whites are.

media exposure, bad luck residence, occupation, income, culture

Page 20: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Correlation Does Not Imply CausationThere is an even bigger problem created by confounds: People tend not to notice them. This leads people to make erroneous causal assumptions from correlational findings, often resulting in unfounded policies, laws, educational practices, and so on. To avoid this, it is essential to keep in mind that correlation does not imply causation.Real-life observation Assumption at the time Alternative explanationStudents of “warm” elementary teachers are more responsive.

Teachers should be trained to have warmer personalities.

Responsive students bring out warmth from their teachers.

Students in private schools outperform those in public schools.

The government should pay for vouchers to allow any student to attend private school who wishes to do so.

The type of student whose parents pay for private school will tend to do bet-ter than others no matter what school is attended.

Mainstreamed special education students show greater achievement than non-mainstreamed special education students.

All special education students should be mainstreamed.

Special education students who are are already high achievers are the ones being mainstreamed.

Page 21: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Reasons for resultsWhen a relationship is found between variables, it could be due to any or all of the four reasons listed below. As an example, consider different reasons that a study of seat-ing placement’s effect on academic achievement may indicate that students who sit up front do better in school.Reason for Difference Explanation Seating exampleIndependent Variable The independent variable

affects the dependent variable (as hypothesized).

Sitting in front helps people learn better.

Extraneous Variables Random factors affect the dependent variable. (possibly a type I error)

The students who sat in front coincidentally were better students on average.

Confounding Variables: Preexisting differences between groups

The groups were already different beforehand.

Students who choose to sit in front usually are better students in the first place.

Confounding Variables: procedural artifacts

The way the experiment was run affected the groups differently.

The experimenter is biased to give higher scores to those in front.

Page 22: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Common causes of procedural artifactsConfounds due to preexisting differences can be eliminated by using within-partici-pants designs or random assignment to condition, but confounds due to procedural artifacts, such as those below, must be carefully avoided by the researchers.Source Definition Result How to avoidPlacebo Effect effect of partici-

pants’ expectations about manipulation

Participants change regardless of manip-ulation.

Use a control group as a comparison.

Experimenter Expectency Effect

effect of experi-menters’ expecta-tions about manipu-lation

Participants change due to experiment-er.

Make experimenters blind to condition.

Experimenter Bias bias by experiment-ers, such as in sub-jective data

Manipulation ap-pears to have larger effect.

Make experimenters blind to condition.

Social Desirability Bias

desire by partici-pants to look good

Responses are bet-ter than reality.

Use neutral word-ing.

Sequence Effects influence of previ-ous data collection

Scores on later trials are different.

Use counterbalanc-ing.

Carelessness or lack of foresight

failure to consider all factors

Unpredicted factors influence results.

Do a pilot study.

Page 23: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Waber, Shiv, Carmon, & Ariely (2008)Participants: 82 adultsProcedure: Participants rated the intensity of shocks of various levels, and then re-peated this after they took a pain relief pill that was actually a placebo.Independent Variable: claimed price per pill: $2.50 or discounted to $0.10Dependent Variable: pain reductionResults: People felt greater reduced pain when they believed the pill was expensive.

$0.10 $2.50

100%

75%

50%

25%

0% $0.10 $2.50

claimed price

prop

ortio

n fe

elin

g re

duce

d pa

in

Page 24: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Rosenthal84 & Jacobson (1966)Participants: all students in an elementary school (3 classes each of grade 1-6)Procedure: Students were given an intelligence test. Twenty percent of the students were chosen randomly, and their teachers were told that the “test for intellectual blooming” indcated that they would make great academic gains that year.Independent Variable: whether or not students were reported to be expected to “bloom” during the yearDependent Variable: change in IQ during the yearResults: In the early grades, the false prediction became a self-fulfilling prophecy.

30

25

20

15

10

5

0 1 2 3 4 5 6

grade level

incr

ease

in IQ

controlpredicted to bloom

Page 25: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Common Sequence EffectsSequence Effect Example of DV that could be affectedFatigue time to run a milePractice number of successful free throwsBoredom average rating of 200 company logosInterference number of words rememberedKnowledge of Study estimate of how many people were in the background

In a within-participants ex-periment, each participant takes part in the differ-ent conditions in a random order. This is called counter-balancing, and can cancel mild sequence effects.

Page 26: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

When different designs are usedIf the independent variable is not categorical, a correlational design is used. Otherwise, an experimental or quasi-experimental design is used, based on the criteria below.

Experimental Quasi-ExperimentalWithin-Participants

It is feasible for participants to take part in each condition, and expected sequence effects are minimal and can mostly be canceled by counterbalancing.

The study involves a before-and-after scenario that cannot easily be done in reverse order.

Between-Participants

Random assignment to condition is feasible, but having each participant take part in multiple conditions is not possible or could cause significant sequence effects.

Randomly assigning condition is impossible (e.g., race), impractical (e.g., high school attended), or unethical (e.g., montly alcohol consumption).Or, participants are already conveniently in groups, such as classes of students, and the groups are presumed to be similar with respect to the dependent variable.

Page 27: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Choosing an experimental or quasi-experimental designTrue experiments are better than quasi-experiments because they eliminate confounds due to preexisting conditions and thus establish causation if there are not other confounds. However, it is not always ethical, practical, or even possible to randomly assign partici-pants to condition.Within-participants designs are better than between-participants designs because each par-ticipant is compared to himself or herself, reducing variability and therefore making type II errors less likely. However, it is not always ethical, practical, or even possible for participants to take part in each condition. In addition, sequence effects may be prob-lematic.

Page 28: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Factorial DesignsA factor is an independent variable, or anything that affects something else. A factorial design involves more than one factors. Such designs are labeled k1 x k2 x k3 x …, where ki is the number of levels of factor i.For example, Rosenthal84 & Jacobson (1966) had two factors: predicted to bloom (yes or no) and grade level (1, 2, 3, 4, 5, or 6). This made it a 2 x 6 factorial design with a total of 12 different conditions. In this case both factors were between-participants, but factorial designs can also have one or more within-participants factors.Factorial designs test not only the effect of each factor individually, like nonfactorial designs, but also interactions between factors.Effect Description Example in R&J (1996)Main Effect There is a significant difference

between levels of a single inde-pendent variable regardless of the other independent variables.

Overall, kids who were predicted to bloom averaged bigger IQ increases than those who were not predicted to bloom.

Interaction The difference between levels of one independent variable varies based on the level of another independent variable.

The increase in IQ by those predicted to bloom was much greater in the younger kids.

Page 29: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Post Hoc analysisPost hoc is latin for “after the fact.” Given that a p value is defined as “the probability of a predicted event happening by chance…,” p values for nonpredicted events do not have meaning and should not be interpreted as significant findings. Doing such post hoc analysis creates a huge likelihood of type I errors, unless advanced statistical proce-dures are applied to compensate.Source Definition Example Result To avoidData Snooping

making hy-potheses based on data and then us-ing the same data to test the hypotheses

Blondes scored higher than brunettes on the essay section of the test, so blondes are better than brunettes at essays.

Coincidenc-es appear to be real find-ings.

Collect new data to test the new hypotheses.

Anecdotal Bias

overreliance on informal information about individual situations

Jarrod has lesbian parents and is struggling in school, so same-sex parents are bad for kids.

Individual situations are given more weight than actual statistics.

Do not make conclusions from anecdotal evidence.

Page 30: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

EthicsIn earlier decades of psychological research, universal ethical guidelines were not estab-lished and some research was highly questionable. Now there are APA ethical guide-lines for both human and animal research.• ethical principles for human research

informed consent formsMilgram’s and Zimbardo’s controversial studies

• ethical principles for animal research

Page 31: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

APA ethical principles for human researchPrinciple DescriptionPreapproval Research proposals are approved by the appropriate

Institutional Review Board (IRB) prior to data collection.

Informed Consent Researchers disclose as much as possible in advance, and acquire participants’ written consent to participate.

Allowed withdrawl Participants can remove themselves or their data from a study at any time with no penalty.

Limited deception Deception is only used when justified by the possible benefits of the research and the research is not possible otherwise.

Limited harm No lasting physical or psychological harm is permitted.Debriefing As soon as reasonable, participants are informed of all

relevant aspects of the research.Confidentiality All data are kept confidential, except as specified in the

informed consent.Honest reporting Data are measured, analyzed, and reported without bias.

Page 32: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Informed Consent FormsComponent DescriptionPurpose the goal of the researchProcedure what to expect (but without revealing anything that will

influence results)Right to withdraw explicit statement that participants may remove them-

selves or their data at any point during or after the studyLimits to confidentiality any instances in which data would not be kept

confidentialPossible risks and benefits

forseeable risks or beneifits for the participants, or societal benefits

Contact information whom to contact for further information

Page 33: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

Two famously semi-ethical studiesStanley Milgram46 (Milgram (1963) and later works) conducted a series of experi-ments involving participants believing they were administering severe electric shocks to another participant.Phil Zimbardo (Haney, Banks, & Zimbardo (1973)) set up a simulated prison and randomly assigned participants to play the role of either guard or prisoner in what be-came known as the Stanford Prison Experiment.

Page 34: Chapter Two Research - ewyner.com 2/Psychology Slides 2.… · (Cheung & Rensvold, 2002). The use of ˙CFI is superior to the use of ˙˛2 as a test of invariance because ˙CFI is

APA ethical principles for animal researchPrinciple DescriptionJustification Research proposals are approved by the appropriate

Institutional Animal Care and Use Committee (IACUC).

Animal knowledge The researchers handling the animals have ample knowledge about their natural state and behaviors.

Care and housing The animals are provided humane care and are kept in healthful conditions that meet or exceed USDA guidelines and are inspected semiannually.

Acquisition Acquisition and transport of animals is done humanely and lawfully.

Limited pain Pain and discomfort is minimized, including by anesthesia and euthanasia as needed.