Hungarian Validation of Zimbardo Time

Perspective Inventory

Gábor OROSZ University of Szeged,

Institute of Psychology 07/10/2011

First of all, the question arises:

„Why is time perspective an interesting topic?”

http://www.youtube.com/watch?v=A3oIiH7BLmg

Validity and reliability of translated questionnaires

•  TP is an interesting topic – but how to measure it? •  Criteria of questionnaire validation:

– Internal consistency – Cronbach’s alphas and test-retest reliability (several days –> years)

– Construct validity – firstly: Exploratory Factor Analysis (EFA); later: Confirmatory factor analysis (CFA) (same vs. different samples)

– Convergent validity: correlation with other variables

– Discriminant validity: Sensation seeking vs. Present hedonism

Criteria of validity and reliability

•  EFA criteria: minimal loading .32 and no cross-loading above .32 (Tabachnik & Fidell, 2001)

•  Cronbach’s alpha at least .7 but .8 is better (Nunnally, 1978) (however, on the basis on my experiences .9 or above alpha measures a very narrow psychological construct)

•  CFA criteria: RMSEA ≤ .06, CFI ≥ .95, TLI ≥ .95 Hu & Bentler (1999)

•  Inter-item correlation: between .15 and .50 (Clark & Watson, 1995)

Some trade-offs of validation 1.  Cronbach’s alphas ↑ with the number of

items and ↓ with the number of factors 2.  Generally, EFA factor structure becomes

clearer by dropping items 3.  CFA is very sensitive to cross-loadings*

and lower factor loadings (↓ than .5) → only a few items meet this criteria

4.  CFA is sensitive to the high N of factors → higher possibility of cross-loadings*

* Inappropriate covariances

The original questionnaire: ZTPI (Zimbardo & Boyd, 1999, JPSP)

Sample: NEFA = 606, NCFA = 361, Age = 19.9 Nitems = 56 Total explained variance = 36%

EFA & α PAST - PAST + PRES HEDON

PRES FATAL

FUTURE

N° OF ITEMS

9 10 15 9 13

EXP VAR. 4,5% 12,3% 8,9% 3,9% 6,3%

Cr α .80 .82 .79 .74 .77

CFA: χ2/df = 2.3 RMSEA = ?, CFI = ?, TLI = ?

ZTPI validations in different cultures I.

Nation Sample Exp. var.

EFA and reliability

Past pos.

Past neg

Pres hed

Pres fat

Fut. χ2/df RMSEA

CFI TLI Authors

USA N = 606 Age = 19.9

N = 56 36%

N of items: Exp. var.: Alpha:

9 4.5% .80

10 12.3% .82

15 8.9% .79

9 3.9% .74

13 6.3% .77

2.3 ? ? ? Zimbardo & Boyd (1999)

Italy N = 1507 Age = 34.7

N = 21 31%

N of items:

Exp. Var.: Alpha:

― ― 8

8.4%

.54

5 7.3%

.49

9 15.1%

.67 ? ? ? ? D’Alessio et al.

(2003)

France N = 419 Age = 21.5

N = 20 33%

N of items: Exp. var.: Alpha:

8 4.4% .70

9 8.1% .72

18 10.5% .79

7 3.7% .70

12 6.1% .74

2.04 .055 ? ? Apostolidis & Fieulaine (2004)

Mexico N = 300 Age = 31.8

N = 56 ―

N of items: Exp. var.: Alpha:

20 ?

.77

25 ?

.80

11 ?

.75 ? ? ? ? Corral-Verdugo

et al. (2006)

ZTPI validations in different cultures II.

Nation Sample Exp. var.

EFA and reliability

Past pos.

Past neg

Pres hed

Pres fat

Fut χ2/df RMSEA

CFI TLI Authors

Spain N = 756 Age = 40,1

N = 56 34%

N of items: Exp. var.: Alpha:

8 4.4% .70

14 11.2% .80

14 7.7% .79

9 4.0% .64

11 6.5% .74

? ? ? ? Diaz-Morales (2006)

Brazil N = 247 Age = 22.5

N = 38 31%

N of items:

Exp. Var.: Alpha:

6 ?

.60

7 ?

.60

9 ?

.55

6 ?

.46

10 ?

.67 ? ? ? ? Milfont et al.

(2008)

Lith. N = 1244 Age = 30,9

N = 56 33%

N of items: Exp. var.: Alpha:

8 3.3%

.70

10 7.7%

.72

14 7.3%

.79

10 4.4%

.70

13 12%

.74 2.22 .044 .67 .65 Liniauskaite &

Kairys (2009)

Taiwan N = 420 Age = 20,5

N = 20

N of items: Exp. var.: Alpha:

4 ?

.68

4 ?

.76

4 ?

.68

4 ?

.49

4 ?

.68 2.01 .05 .93 - Gao (2011)

Hungarian validation of ZTPI I.

•  Main goals: 1.  Achieving an appropriate EFA factor

structure (e.i. the rule of .32) 2.  Achieving appropriate internal consistency in

terms of higher alphas than .7 3.  Achieving appropriate CFA (RMSEA, CFI, TLI)

•  Not goals (yet) 1.  Test-retest reliability 2.  Convergent validity & discriminant validity

Hungarian validation of ZTPI II. •  Translation:

–  Five persons translated independently the original ZTPI to Hungarian (3 psychology MA students who knew the topic and have at least advanced language exams, and 2 English teachers who did not know the topic and who have MA in English)

–  Then all of us discussed the inconsistencies until finding the best solution

–  The final solution was given to a bilingual psychology student for backtranslation to Hungarian

–  Finally, the seven persons discussed the final solution

Sample •  1364 persons

–  924 women and 405 men –  Age between 14 and 86 (M = 32.19, SD = 14.70)

•  Education: –  142 primary school –  53 vocational school –  399 high school degree –  712 higher education (BA, BSC, MA, MSC) –  51 postgraduate degree

•  Place of residence: –  170 villages, 553 towns, 326 county towns, 309 capital

Measures •  The translated version of ZTPI (56 items), 5-point Likert

scale (1 = very uncharacteristic; 5 = very characteristic) •  Gender •  Age •  Marital status •  Place of residence •  Perceived financial status •  Expected financial situation in 5-10 years •  Level of education •  Perceived health

Results 1: EFA & CFA •  Principal Axis Factoring (PAF) extraction •  Promax rotation (Kappa = 0) (Brown, 2006)

– Oblique solution instead of orthogonal: (1) it provides better results for the CFA, (2) TP subscales correlated in previous studies (Anagnostopoloulos & Griva, 2011)

– Respecting the rule of .32 of Tabachnik & Fidell (2001)

A 36 item solution emerged: Strong points •  Five factors: scree test •  Exp. Var: 45.9% •  Bartlett and KMO: OK •  Alphas are higher than .7

Weak points CFA results:

•  RMSEA = .068 !!! (X < .06) •  CFI = .75 !!! (X > .95) •  TLI = .72 !!! (X > .95)

The final 17-item version

RMSEA = .039 CFI = .963 TLI = .954

36 items ↓

17 items

OK, but how the hell can I carry out a confirmatory factor analysis?

CFA – how to do it?

•  The most important message of this presentation:

„USE YOUTUBE IF YOU DON’T KNOW HOW TO CARRY OUT A STATISTICAL ANALYSIS!”

http://www.youtube.com/watch?v=JkZGWUUjdLg

Potential problems… •  It is not the best to use ML method if we don’t have the normal

distribution of our variables (generally it is the case…) •  Modification indices can be very useful in the item selection, do

not overdose it (follow common sense as well) •  Error covariances can be useful, however, we have to explain

why we put such error covariances (case of Vallerand) •  It is very difficult to achieve a good model fit if we have 4-5

factors and 4-5 items per factor •  This method pushes authors to create scales with few items

which can measure efficiently narrow psychological constructs –  TP is a multidimensional phenomena and NOT a narrow

construct…

Are Competition and Extrinsic Motivation Reliable Predictors

of Academic Cheating?

Gábor OROSZ University of Szeged

Institute of Psychology 07/10/2011

Theoretical roots of this question

•  “Competition is perhaps the single most toxic ingredient in a classroom, and it is also a reliable predictor of cheating” (Anderman & Murdock, 2007, p. XIII)

•  Competition has an overall negative impact on performance, problem solving, and personal relationships in comparison with cooperation (Deutsch, 1949, Johnson & Johnson, 1974, 1979, 1982; Johnson, Maruyama, Johnson, Nelson, Skon, 1981; Lewis, 1944a, 1944b; Qin, Johnson & Johnson, 1995)

•  Competition undermines intrinsic motivation (Deci, Betley, Kahle, Adrams & Porac, 1981; Vallerand, Gauvin, Hallivell, 1986a, 1986b)

Paradigm shift of competition •  Solid theoretical basis regarding competition’s positive

consequences: –  It can improve performance, interpersonal

relationships, resource control and intrinsic motivation (Bornstein, Erev & Rosen, 1990; Carnavale & Probst, 1997; Charlesworth, 1996; Epstein & Harackiewicz, 1992; Erev et al., 1993; Fülöp, 1997, 1999, 2001, 2004; Harackiewicz, 1998; Hawley, 2003, 2006; Hurlock, 1927; Moede, 1914; Reeve & Deci, 1999; Ryckman, Hammer, Kaczor & Gold 1996; Sims, 1927; Tauer & Harackiewicz, 2004; Tassi & Schneider, 1997; Tjosvold, Johnson, Johnson & Sun, 2003, 2006; Young, Fisher & Lindquist, 1993; Wentzel, 1991; Whittemore, 1924; Julian, Bishop, & Fiedler, 1966; Rabbie, & Wilkens 1971; Reeve, Cole & Olson, 1986; Reeve & Vallerand, 1984; Vallerand & Reid, 1984)

–  Furthermore, perceived classroom competition was positively related to self-reported cheating behavior (Smith, Ryan & Diggins, Taylor, 1972; Pogrebin & Dodge, 2002 Whitley, 1998)

The nature of paradigm shift of competition

New paradigm Competition is not

opposed to cooperation

Competition is a heterogenous phenomena

Competition is a situational and personality variable

Old paradigm Competition is opposed

to cooperation

Competition is a homogenous phenomena

Competition is a situational variable

First goal of the present study

•  To assess the impact of individual-level and situation-level competition-related variables on academic cheating.

–  Aggressively competing students will cheat more, than those students who have positive attitudes towards competition.

–  Such classroom atmosphere in which the goal is achieving the recognition of the teachers will induce more cheating, than such competitive classroom atmosphere which promotes skill development.

Second goal of the study

–  Previous studies found that mastery and intrinsic motivations reduce cheating, whereas performance goals and extrinsic motivations enhance it. (Anderman, Griesinger, & Westerfield, 1998; Anderman & Midgley, 2004; Anderman & Murdock, 2007; Murdock & Anderman, 2006)

–  In these studies performance goal orientation and competitive pressures are interpreted as overlapping concepts – NO DISTINCTION BETWEEN THE TWO

•  Distinction between the effects of motivation- and competition-related factors’ of cheating.

Third goal of the study •  Comparing the relative importance of motivational

and competition-related variables with proximal variables of cheating behavior such as:

(1) attitudes towards cheating (3) risk of detection (2) guilt (4) possible punishments

We hypothesize that the importance of motivational and competition-related factors is overrated in the literature of academic cheating

More proximal factors have vastly larger impact of cheating than extrinsic motivation and competition

Factors that have effect on cheating I.

•  Grade point average – negative (Kerkvliet, 1994; Kerkvliet & Sigmund, 1999; Leming, 1980; Newstead, Franklyn-Stokes & Armstead, 1996; Whitley, 1998; Straw, 2002)

•  Attitudes towards cheating – positive (Bolin, 2004; Jordan, 2001; Jensen, Arnett, Feldman & Cauffman, 2001; Whitley, 1998)

•  Guilt – negative (Diekhoff, LaBeff, Shinohara & Yasukawa,1999; Malinowski & Smith, 1985)

•  Classroom competition – positive (Anderman & Murdock, 2007; Smith, Ryan & Diggins, 1972; Taylor et al., 2002)

•  Self-developmental competition – negative (Orosz, 2010)

•  Hypercompetitive traits – positive (Orosz, Jánvári, Salamon, 2011)

Factors that have effect on cheating II.

•  Motivation: Vallerand et al., 1992 – Academic Motivation Scale –  Intrinsic motivation to know – negative –  Extrinsic motivation of external regulation – positive –  Amotivation - positive

•  Risk of detection – negative (Heisler, 1974; Leming, 1978; Corcoran & Rotter, 1987; Covey, Saladin & Killen, 1989; Whitley, 1998)

•  Expected punishments – negative (Bunn, Caudill & Gropper, 1992; Cohran, Chamlin, Wood & Sellers, 1999)

Hypotheses I. H1a: Attitude of aggressive competition will be positively

correlated with academic cheating H1b: Attitude of self-developmental competition will be

negatively correlated with cheating H1c: Positive attitude towards competition will be unrelated

to cheating H1d: Such classroom climate, in which the goal of

competition is recognition by the teachers, will be linked with academic dishonesties

H1e: Competitive climate, which promotes self-development, will lead to lower prevalence of cheating

Hypotheses II.

H2: Motivational pattern (IM, EM, AM) are separate from those of the competition-related individual and contextual variables

H3: the magnitude of the effects of motivational and competition-related variables on academic dishonesty is lower than those of the proximal individual (GPA, attitudes towards cheating, guilt) and situational variables (risk of detection, expected punishments)

Participants •  620 high school students (M = 264, F = 356)

•  19 classes from 7 schools – 2 schools upper third, 3 middle third, 2 lower third section of Hungarian high school ranking

•  Age: 13-20 years, M = 16.66 years (SD = 1.51)

•  Teachers were not present during data gathering

•  381 students filled in the questionnaire concerning competitive climate, 236 students did not fill in this scale

Measures - individual •  Individual differences of competition:

–  Aggressive competition scale: „I can be aggressive with my rivals” or “I’m often in conflict with my opponents”

–  Self-developmental competition scale: “Competition helps me to improve my skills” or “Competition brings the best out of me”

–  Positive attitudes towards competition scale: “I like the challenge of competition” or “Competition inspires me”

•  Individual differences of motivation Vallerand et al. AMS: –  IM to know – motivation to acquire knowledge –  EM external regulation – learning due to only external pressures

and obligations –  AM – the absence of motivation

Measures – proximal & situational •  Two cheating vignettes: cheating sheet & copying

–  self-reported cheating –  acceptance –  punishments –  feeling of guilt –  Perceived risk of detection

•  Competition Climate Scale (CCS): –  Constructive competition (CC) - it has positive impact on

students performance, creativity, interpersonal relationship –  Destructive competition (DC) – it has negative impact on

relationships, the goal is achieving the recognition of teachers

Results - Descriptives Self-

reported cheating

Accept. Guilt Risk of

det. Exp.

punish.

Cheating sheets

no 24.6%

1 4.4% 40.5% 2.4% 14.4%

2 33.8% 37.1% 14.7% 82.6%

yes 75.4% 3 47.1% 17.5% 72.0% 2.5%

4 14.8% 4.9% 10.9% .5%

Copying no 38.1%

1 7.6% 35.3% 1.3% 12.1%

2 39.3% 33.2% 7.8% 86.2%

yes 61.9% 3 41.5% 21.1% 59.3% 1.2%

4 11.5% 10.4% 31.6% .5%

χ2/df = 1.786, CFI = .959, TLI = .952, RMSEA = .036

Results on the basis of the model I.

H1a: Aggressive competition has a positive indirect effect on SR cheating – proved

H1b: Self-developmental competition has a negative indirect effect on SR cheating – proved

H1c: Positive attitude towards competition is unrelated to cheating – proved

H1d: Such classroom climate, in which the goal of competition is recognition by the teachers, will be linked with academic dishonesties – not proved: DC climate is unrelated

H1e: Competitive climate, which promotes self-development, will lead to lower prevalence of cheating – not proved: CC climate is unrelated

Results on the basis of the model II.

H2: Motivational pattern (IM, EM, AM) are separate from those of the competition-related individual and contextual variables – proved: both CFAs and the model indicates that

H3: the magnitude of the effects of motivational and competition-related variables on academic dishonesty is lower than those of the proximal individual (GPA, attitudes towards cheating, guilt) and situational variables (risk of detection, expected punishments) – proved: (1) competitive climate (CC & DC) has no effect, (2) SD Comp & Aggr. Comp has small indirect effect, (3) extrinsic motivation no effect, BUT IM & AM have a serious effect!

Discussion “Competition is perhaps the single most toxic ingredient in a classroom, and it is also a reliable predictor of cheating” (Anderman & Murdock, 2007, p. XIII) – probably not true!

1.  It is not the extrinsic motivation but the amotivation which counts!

2.  Intrinsic motivation can prevent cheating!

3.  Both motivational and competition-related effects are less important than proximal variables such as acceptance, GPA and guilt

− Are Competition and Extrinsic Motivation Reliable Predictors of Academic Cheating?

− Not really!!!

Practical implications •  Eliminating competition from classroom is not the

best way to prevent cheating! (Tjosvold, Fülöp, etc) •  Eliminating extrinsic motivation is not the best way

either in order to prevent cheating! (see Haraczkiewicz, Pintrich, etc.)

•  Eliminating amotivation and increasing intrinsic motivation can be more useful! (enthusiasm studies)

•  As teachers we have to create such environment during exams in which we prevent students from cheating: Risk of detection is more important than serious punishments (Houston)

Limitations & further directions •  Hungarian educational context: in other contexts high

school competition can be more destructive which creates more conflicts, which induce individual cheating and prevent from collaborative cheating

•  In Hungary a pretty large proportion of students cheated. It is surely different in other countries (i.e. France )

•  It would be important to measure SR cheating in a more gradual way (occurrence of cheating per semester)