personality retest effects: guilt as a mechanism for managing response distortion
DESCRIPTION
Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion. Jill E. Ellingson, Eric D. Heggestad, and Erin E. Coyne October 13 – 14, 2006 ETS Technical Advisory Group Meeting. Current Retesting Policy. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/1.jpg)
Personality Retest Effects: Guilt as a Mechanism for
Managing Response Distortion
Jill E. Ellingson, Eric D. Heggestad, and Erin E. Coyne
October 13 – 14, 2006ETS Technical Advisory Group Meeting
![Page 2: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/2.jpg)
Current Retesting Policy
• Applicants allowed to voluntarily retake assessment after period of time if displeased with outcome– Applicant elects to retake the assessment
• Common in organizations which use assessment tools for hiring
• Most often used in conjunction with cognitively-loaded assessments
![Page 3: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/3.jpg)
Personality Assessment Retesting• Organization directs certain applicants whose
responses are likely distorted to retake the personality assessment
• Responses deemed distorted on basis of embedded intentional distortion scale– Flags extreme response profiles
• Applicants informed that responses were flagged as suspect
• Hiring decisions made using retested scores
![Page 4: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/4.jpg)
Key Questions
• Does retesting flagged applicants lower previously inflated personality scale scores?
• What psychological mechanism operates within applicants to help explain why they would adjust their responses?
![Page 5: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/5.jpg)
Hypothesis 1: Retesting flagged individuals will result indecreased personality scale scores in the secondassessment relative to the first assessment.
Scale Score Changes
• Flagged applicants have positively biased score profiles• Retest effect evident in degree to which second
assessment scores are lower• Preliminary research suggests that scores may be
lowered up to 0.7 standard deviation units (Ellingson & Heggestad, 2003)
![Page 6: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/6.jpg)
Role of Guilt: Appraisal Theory
EventEvaluation Factors
Relevance?
Congruence?
Associated values?
Accountability?
Coping potential?
Behavior
Emotion
![Page 7: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/7.jpg)
Role of Guilt: Applicant Appraisal
Told toretest
Evaluation Factors•Personally relevant
•Incongruent•Violates personal standards
•Personally accountable
•Coping potential?
Guilt
BehaviorHypothesis 2: Retesting flagged individuals willresult in increased feelings of guilt in the secondassessment relative to the first assessment.
![Page 8: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/8.jpg)
Role of Guilt: Applicant Appraisal
Told toretest
Evaluation Factors•Personally relevant
•Incongruent
•Violates personal standards
•Personally accountable
•Make reparation
Guilt
Hypothesis 3: The level of guilt reported by flagged individuals in the second assessment will moderate the degree to which personality scale scores change.
Respondhonestly
![Page 9: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/9.jpg)
Sample and Measures
• 288 undergraduate students• Measures:
– NEO-Five Factor Inventory (NEO-FFI)– Balanced Inventory of Desirable Responding-
Impression Management scale (BIDR-IM)– Personal Feelings Questionnaire-2 Guilt scale
(PFQ2-G)– Positive and Negative Affect Schedule-Expanded
(PANAS-X)– Test of Self-Conscious Affect-3 (TOSCA3)
![Page 10: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/10.jpg)
ProcedureAll participants:1. Completed the TOSCA32. Completed NEO-FFI and BIDR-IM
under motivating instructions3. Completed the PFQ2-G and PANAS-X
regarding feelings had while takingthe personality measure
Sorted participantsinto 3 groupsbased on
BIDR-IM score
Low Control Group•Low BIDR-IM score
High Control Group•High BIDR-IM score
Flagged Retest Group•High BIDR-IM score
Retested for neutral reason Told responses weresuspect and unusable
Asked to retest
Tim
e 1
Tim
e 2
![Page 11: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/11.jpg)
Effect Sizes
NEO-FFI Low Control High Control Flagged Retest
Extraversion 0.36 0.29 0.45
Conscientiousness 0.04 0.22 0.59
Agreeableness 0.12 0.26 0.56
Openness 0.04 0.10 0.28
Emotional Stability -0.10 0.05 0.27
Average d 0.09 0.18 0.43
Impression Management -0.03 -0.13 0.54
State Guilt 0.20 0.33 0.07
Positive values indicate that Time 1 score was larger than Time 2 score.
![Page 12: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/12.jpg)
Repeated-measures MANCOVA:Personality Scales
Source Pillai’s Trace F df p η2
Within-subjects effects
Time 0.018 1.045 (5, 280) 0.392 0.018
Time x Trait Guilt 0.010 .577 (5, 280) 0.718 0.010
Time x Condition .170 5.232 (10, 562) 0.000 0.085
Between-subjects effects
Trait Guilt 0.093 5.739 (5, 280) 0.000 0.093
Condition 0.255 8.229 (10, 562) 0.000 0.128
Within-subjects factor (Time): Time 1, Time 2Between-subjects factor (Condition): Low Control, High Control, Flagged Retest
Covariate: Trait Guilt
![Page 13: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/13.jpg)
Repeated-measures ANCOVA:Personality Scales
Within-subjects effects Between-subjects effects
Time Time x Guilt Time x Condition Trait Guilt Condition
NEO-FFI F η2 F η2 F η2 F η2 F η2
Extraversion 2.32 0.008 0.06 0.000 2.20 0.015 1.10 0.004 0.75 0.005
Conscientiousness 0.95 0.003 0.01 0.000 16.78* 0.106 4.02* 0.014 17.25* 0.108
Agreeableness 1.30 0.005 0.01 0.000 7.84* 0.052 1.24 0.004 33.07* 0.189
Openness 2.62 0.009 1.01 0.004 6.61* 0.044 8.21* 0.028 0.38 0.003
Emotional Stability
2.45 0.009 1.55 0.005 12.59* 0.081 6.26* 0.022 7.94* 0.053
Within-subjects factor (Time): Time 1, Time 2Between-subjects factor (Condition): Low Control, High Control, Flagged Retest
Covariate: Trait Guilt
* p < .05
![Page 14: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/14.jpg)
Repeated-measures ANCOVA:Agreeableness Interaction
48
50
52
54
56
58
60
62
64
66
68
Tim e 1 Tim e 2
Ag
reea
ble
nes
s sc
ale
mea
ns
High Control
Flagged Retest
Low Control
![Page 15: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/15.jpg)
Repeated-measures ANCOVA: State Guilt
Source Pillai’s Trace F df p η2
Within-subjects effects
Time 0.004 1.274 (1, 283) 0.260 0.004
Time x Trait Guilt 0.002 .466 (1, 283) 0.495 0.002
Time x Condition 0.004 .537 (2, 283) 0.585 0.004
Between-subjects effects
Trait Guilt 81.353 4.500 (1, 283) 0.035 0.016
Condition 244.106 6.752 (2, 283) 0.001 0.046
Within-subjects factor (Time): Time 1, Time 2Between-subjects factor (Condition): Low Control, High Control, Flagged Retest
Covariate: Trait Guilt
![Page 16: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/16.jpg)
Moderated Regressions:Understanding Score Change
Full Model Predicting Time 2 Personality Scale ScoresStandardized Beta Coefficients and Variance Explained
Predictors Extraversion Conscientiousness Agreeableness OpennessEmotional
Stability
Step 1 Time 1 Score .838* .888* .829* .910* .850*
Step 2 Time 2 State Guilt -.083 -.195* -.291* -.138* .141*
Low Control Condition .097 .176* .064 .106* -.163*
High Control Condition .087 .143* .101* .085* -.114*
Step 3 Guilt x Low Control .074 .104* .225* .135* -.072
Guilt x High Control .077 .123* .098* .058† -.077†
R-squared .688 .730 .726 .826 .722
Δ R-squared .004 .010* .018* .007* .004
* p < .05† p < .10
![Page 17: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/17.jpg)
Moderated Regressions:Conscientiousness Interaction
Moderating Effect of State Guilt onTime 2 Conscientiousness Scores
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
Low State Guilt High State Guilt
Tim
e 2
Co
ns
cie
nti
ou
sn
es
s
Sc
ore
Low Control
High Control
Flagged Retes t
![Page 18: Personality Retest Effects: Guilt as a Mechanism for Managing Response Distortion](https://reader036.vdocument.in/reader036/viewer/2022062410/56815a30550346895dc77105/html5/thumbnails/18.jpg)
Conclusion
• Retesting flagged applicants will result in a set of personality scale scores that are less positively inflated
• The appraisal profile of guilt helps explain this effect– Flagged applicants who feel guilty as a result
of being retested decrease their scores in response.