Training and Experience

Dr. Mike AamodtRadford University

maamodt@radford.eduUpdated 12/06/2004

Personnel Selection Methods• Training & Education• Experience

– Applications/Resumes– Biodata– Interviews

• Knowledge• Ability

– Cognitive– Physical– Perceptual

• Skills– Work Samples– Assessment Centers– References

• Personality & Character– Personality Tests– Integrity Tests

• Medical– Medical Exams– Psychological Exams– Drug Testing

Ratings of Training

• Education

• Work-Related Training

• Military

Education as a Predictor of Job Performance

• Education can be broken further into 4 categories:– Degree

– Years of education

– Major

– GPA

Does Education Predict

Performance?

Summary of Meta-Analyses

Meta-analysis Occupation K N ρ

Aamodt (2004) Police 54 9,120 .28

Vineberg & Joyner (1982) Military 35 .25

Hunter (1980)

Hunter & Hunter (1984)

Schmidt & Hunter (1998)

USES data base

425 32,124 .10

Dunnette (1972) Entry level petroleum

15 .00

Validity of Education for PoliceOur Updated Meta-Analysis

Academy Supervisor Discipline

Grades Ratings Problems

________ _________ ________

Number of studies 32 54 54

Sample Size 6,153 9,120 21,416

Uncorrected validity .26 .17 - .08

Corrected validity .38 .28 - .12

Sampling error % 100% 80% 73%

Our Meta-Analysis (Cont.)

Academy Grades Patrol Performance_____________ _______________

Education level N Score N Score______________ _____ _____ _____ _____GED/HS Diploma 376 -.23c 915 -.12a

12-64 college hours 100 -.05ac 391 .02b

AAS 98 .16ab 139 .07bc

64 college hours 62 .01ac 45 .29bc

Bachelor’s 255 .32b 441 .18c

Interaction of Education and ExperienceRatings of Patrol Performance

Experience

Educational Level < 2 years 2+ Years Total

High School Diploma M - .09ab - .16a - .13a

SD (.92) (.98)

Some college or associate’s degree

M - .26a .07b - .09a

SD (1.02) (.94)

B.A. or higher M .04ab .34c .19b

SD (1.02) (1.12)

Education and Academy GradesEffect of Academy Length

K N r ρ SE %

Overall 32 6,153 .26 .38 100

Academy Length

12-17 weeks 16 2,477 .31 .46 100

> 20 weeks 15 3,473 .25 .36 100

Does Education Add Incremental

Validity Over Cognitive Ability?

Education and Incremental Validity

• Schmidt & Hunter (1998) say no

– Cognitive ability (r = .51)

– Cognitive ability and education (r = .52)

Education and Incremental Validity

• Our meta-analysis for police says yes

– Academy Grades

• Cognitive ability (r = .33)

• Cognitive ability and education (r = .34)

– Patrol Performance (uncorrected)

• Cognitive ability (r = .16)

• Cognitive ability and education (r = .29)

Can You Substitute Education for a Cognitive Ability Test?

• Education has less adverse impact• But

– Cognitive ability and education are only slightly correlated

• .29 in our police data• .00 assumption by Schmidt & Hunter (1998) of applicants

applying for a particular job• .55 estimate by Schmidt & Hunter (1998) for the general

population

– Education adds incremental validity for law enforcement

– Our dataset contains plenty of college graduates with low cognitive ability

Does Grade Point Average

Predict Performance?

Validity of GPA• GPA is a valid predictor of performance on the

job, training performance, starting salary, promotions, and grad school performance

• GPA is most predictive in the first few years after graduation (Roth et al., 1996)

• GPA will result in high levels (d=.78) of adverse impact (Roth & Bobko, 2000)

• People with high GPAs– Are intelligent (r=.50; Jensen, 1980)– Are conscientious (r=.34; Bevier et al., 1998)

Validity of GPAMeta-Analysis Results

r ρWork-Related Criteria

Job performance (Roth et al., 1996) .16 .36

Training performance (Dye & Reck, 1989) .29

Promotions (Cohen, 1984) .16

Salary (Roth & Clarke, 1996)

Starting salary .13 .20

Current salary .18 .28

Graduate School Performance (Kuncel et al., 2001)

Grades .28 .30

Faculty ratings .25 .35

Predicting Graduate School Performance

Graduate GPA Teaching Ratings

GRE Score .33* .00

GPA – Overall .31* .02

GPA – Psych .32* .03

GPA – Junior/Senior .40* .09

Faculty References

Mental agility .25* - .08

Vigor - .26* - .07

Extroversion .03 .24*

Agreeableness - .11 .06

Conscientiousness .16 - .04

Does College Major Predict Performance?

College Major as a Predictor of Job Performance

• Aamodt (2004) meta-analysis• Validity of having a criminal justice major___________________________________ r k n

____ ___ _____Patrol performance - .02 9 1,706Academy grades .01 6 976Discipline problems .00 10 2,621___________________________________

Will Education Requirements

Result in Adverse Impact?

Adverse ImpactTechnique White-

BlackWhite-

HispanicMeta-analysis

Cognitive ability 1.10 .72 Roth et al. (2001)

GPA .78 Roth & Bobko (2000)

Biodata .33 Bobko et al. (1999)

Reference Ratings .25 Aamodt (2002)

Structured interview .23 Huffcutt & Roth (1998)

Recommendations .22 Aamodt (2002)

HS diploma .20 .52 Aamodt (2002)

Bachelor’s degree .18 .26 Aamodt (2002)

Personality .09 Schmitt et al. (1996)

Integrity tests .07 -.05 Ones & Viswesvaran (1998)

Lingering Questions• Is the validity of education job specific?• What is the actual incremental validity of

education over cognitive ability?• Why would education predict performance?

– Knowledge

– Liberal arts skills

– Mental ability

– Motivation

Work-Related Training as a Predictor of Job PerformanceAamodt (2000) Meta-Analysis

• Performance in the academy predicts probationary performance (ρ=.44)

• Performance in the academy predicts patrol performance (ρ = .38)

• Performance in the probationary period predicts patrol performance (ρ = .44)

Military Training as a Predictor of Job Performance

• It is common for Police departments to give preferential treatment to candidates with Military experience for two reasons:

– Reward veterans for serving their country

– They believe veterans will be better cops because the training that they’ve gone through and that of police are similar

Validity of Military Experience and Police Performance

Academy Grades

Patrol Performance

Commendations

K 9 16 8

N 2,061 4,090 1,337

r .02 - .03 - .07

ρ .04 - .05 - .10

SE % 90% 23% 81%

Military Experience in Candidate Selection

• In selecting applicants, the employer must be careful because:– Minorities tend to have a higher rate of undesirable

discharges; therefore any questions about military service dates, type of discharge, and discipline received while in the service can result in adverse impact.

- Gatewood & Feild (2001)

Experience

Experience

• Past behavior predicts future behavior– Experience is a valid predictor of future

performance (r = .27; Quinones et al., 1995)

• Types of Experience– Work

– Life

Experience

• Evaluated through:– Application blanks

– Resumes

– Interviews

– Reference checks

– Biodata instruments

Experience

• Considerations– How much experience?– How well did the person perform?– How related is it to the current job?

Experience Predicts Best…

• Credit prior work experience only:– In the same occupational area as that in which performance is to

be predicted

– In the performance of tasks or functions that have direct application on the job

• Recency of experience should be used as a decision rule for awarding credit only when justified on a case-by-case basis

• Credit for duration of work experience should be limited to a few years.

• High prediction up to about 3 years of experience, declining to low prediction for more than 12 years of experience.

Experience for Selection : Some Concerns

• Sullivan (2000) claims that “experience in solving ‘past problems’ is rapidly losing its applicability to current and future problems.”

• Organizations will increase their applicant pool if they delete the “ancient history” requirements (i.e. “Ten years experience required”).

Sullivan (2000)

1) Reduce or eliminate the number of years required in your ads and replace them with “the demonstrated ability to solve problems with our required level of difficulty.

2) Use simulations and actual problems to assess applicants.3) Develop “future-oriented” questions for applicants.4) Train evaluators and compensation professionals to put

less weight on experience of candidates.5) Revise job descriptions to include level of difficulty.6) Identify the amount and type of experience and

competencies that would predict job performance.7) Check to see if there is a correlation between the number

of years of experience an employee has and their success in your firm.

Experience: Some More Concerns

• Performance matters• “Haven’t done” doesn’t mean “can’t do”• Experience has a shelf life• Listing something on a resume is not experience• Where you get your experience matters• Experience does not guarantee success• Experience is expensive• More experience might be bad (old ways and

ideas)