MGTO 324

Workshop 6: Criterion Validity

Multiple Regression Analysis

Background You are an HR manager of an insurance firm. The most

salient culture in this organization is that they want their insurance agents to be

creative in helping their clients solve their financial problems. calm and patient in listening and understanding the clients’

needs. Supervisors gave an overall performance rating for each

agent according to the above criteria. Suppose you used two personality scales, Openness to

Experience and Emotional Stability, to predict job performance.

You choose to measure these two personality traits because they are theoretically related to job performance in this context.

You collected the personality data two years ago and the job performance ratings this year.

Criterion-related validity

Try to show criterion-related evidence of the two personality scales

In addition, try to show the value of the personality tests by assessing the incremental validity of each scale

Simple correlation

Correlations

1 .526** .298**. .000 .003

97 97 97.526** 1 .030.000 . .773

97 97 97.298** .030 1.003 .773 .

97 97 97

Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)N

Performance_Rating

Openness_to_Experience

Emotional_Stability

Performance_Rating

Openness_to_Experience

Emotional_Stability

Correlation is significant at the 0.01 level (2-tailed).**.

Scatter Plot

1.00 2.00 3.00 4.00 5.00 6.00 7.00

Openness_to_Experience

1.00

2.00

3.00

4.00

5.00

6.00

Perf

orm

ance

_Rat

ing

r = .526; p < .05

2.00 3.00 4.00 5.00 6.00 7.00

Emotional_Stability

1.00

2.00

3.00

4.00

5.00

6.00

Perf

orm

ance

_Rat

ing

r = .53; p < .05

Linear Regression

Model Summary

.526a .276 .269 1.10853Model1

R R SquareAdjusted R

SquareStd. Error ofthe Estimate

Predictors: (Constant), Openness_to_Experiencea.

ANOVAb

44.601 1 44.601 36.295 .000a

116.740 95 1.229161.340 96

RegressionResidualTotal

Model1

Sum of Squares df Mean Square F Sig.

Predictors: (Constant), Openness_to_Experiencea. Dependent Variable: Performance_Ratingb.

Coefficientsa

1.933 .348 5.553 .000.504 .084 .526 6.025 .000

(Constant)Openness_to_Experience

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: Performance_Ratinga.

PR = 1.933 + .504 x OE; Z(PR) = .526 x Z(OE)

This equation accounts for 27.6% of the PR’s variance, and it is significantly better than 0%.

1.00 2.00 3.00 4.00 5.00 6.00 7.00

Openness_to_Experience

1.00

2.00

3.00

4.00

5.00

6.00

Perf

orm

ance

_Rat

ing

r = .526; p < .05

Z(PR) = .526 x Z(OE)

Multiple regression

Equal to linear regression, except that more than one predictor in the equation PR = Constant + B(OP) x OP + B(ES) x ES

InterpretationCoefficientsa

-.031 .627 -.050 .960.489 .079 .510 6.177 .000.331 .098 .277 3.365 .001.022 .013 .136 1.651 .102.133 .216 .051 .616 .539

(Constant)Openness_to_ExperienceEmotional_StabilityAgeGender

Model1

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: Performance_Ratinga.

• OE and ES are both significant predictors.

• The explained unique portion of the PR’s variance. That is, their predictions could not be replaced by other predictors in the equation, which shows the incremental validity.

Multiple Regression Analysis

Background You are an HR manager of a firm. There are two

performance criteria Salary increase after the first year’s performance appraisal Quit intention

After reviewing some leading research articles in Journal of Applied Psychology, Personnel Psychology, and Academy of Management Journal, you are thinking that perhaps their job satisfaction and job commitment to their previous jobs (i.e., before joining your organization) may predict these two variables

You also have an intuition that the outcome criteria should be related to one’s Extraversion

You measured their job satisfaction, job commitment, and Extroversion before joining your firm

You got the two criteria data one year after performance appraisal

Assignment

Task Show the criterion validity of each predictor Show the incremental validity of each predictor Comments on the usefulness of each predictor Comments on any interesting observation