ReliabilityDefinition: The stability or consistency of a test.

Assumption: True score = obtained score +/- error.

Domain Sampling Model

Item Domain

Test

Reliability Techniques

1. Test-retest

2. Parallel Forms

3. Split-half

T T1 2

A A1 2

TA B

Score Score

1

100

50 pairs

4. Internal Consistency

K-R-20 Coefficient Alpha

Content Validity [the extent to which test items represent the domain]

a) Subject Matter Expert Opinions (e.g., CVR statistic)

b) Internal consistency reliability

c) Correlation with other similar tests

Types of Validity

Predictive –

[Correlation between test scores of applicants and their performance scores when some time interval has passed after they are hired]

• Range restriction issue on performance scores• Time, cost, & pragmatic concerns

Criterion-related Validity

Concurrent --

[Correlation between test scores and performance scores of current employees]

• Motivation level

• Guessing, Faking

• Job experience factor

• Range restriction issue on performance scores

Types of Validity (cont.)

Types of Validity (cont.)

Construct Validity [the extent to which the test assesses the construct it intends to measure]

• Correlation between scores measuring a construct (e.g., anxiety) with one method (e.g., paper & pencil) with scores on the same construct using a different method (e.g., interview) [Convergent validation]

• Correlation between scores measuring a construct (e.g., anxiety) using one method (e.g., paper & pencil) with scores on a different construct (e.g., leadership) assessed with a different method (e.g., interview) [Discriminant validation]

• Application Blanks (Weighted Application Blanks, Biographical Information Blanks)

• Honesty (Integrity) Tests, Drug Tests

• Interviews (e.g., Situational)

• Aptitude Tests (e.g., Mechanical, Clerical)

• Work Samples

• Assessment Centers (Situational Exercises)

Types of Selection Tests

Types of Selection Tests (cont).

•Personality Inventories

Self-reports --

(e.g., Minnesota Multiphasic Personality Inventory (MMPI), California Psychological Inventory (CPI), Myers-Briggs Type

Indicator (MBTI), Personality

Inventory (HPI), NEO PI-R NEO (assesses the 5-Factor Model consisting of: Neuroticism, Extraversion Openness, Agreeableness Conscientiousness)

(e.g., Thematic Apperception Test (TAT), Rorschach Inkblot Test (RIT)

Projective Techniques --

Weighted Application Blank (WAB)

Develop test items

Job Performance

Absenteeism

Low

High

r

r

Select items that relate to

criterion (e.g., absenteeism

Correlate items with performance

Weight items according to their relationship to job

performance

_______

_______

_______

_______

_______

Weighted Application Blank (WAB)

Develop test items

Job Performance

Absenteeism

Low

High

r

r

Select items that relate to

criterion (e.g., absenteeism

Correlate items with performance

Weight items according to their relationship to job

performance

_______

_______

_______

_______

_______

Frequency of Common Inappropriate Application Blank Questions

ItemNot Appropriate

Worded Appropriately

Not Asked

Past salary 98.9% 0% 1.1%

Minimum acceptable salary 72.7% 0% 27.2%

Reference source 59.1% 0% 40.9%

Age 54.5% 37.5% 8.0%

Information about

relatives 50.0% 10.2% 39.8%

Conviction records 43.2% 28.4% 28.4%

Health 40.9% 2.3% 56.8%

Military service 30.7% 30.7% 38.6%

Marital status 27.3% 0% 72.7%

Who to notify in case

of emergency 25.0% 43.2% 31.8%

Length of time in

residence 23.9% 0% 76.1%

Physical description, photo 19.3% 0% 80.7%

Rent or own car or home 18.2% 0% 81.8%

Handicap 17.0% 6.8% 76.2%

Organizational membership 15.9% 21.6% 12.5%

Work schedule 13.6% 63.6% 22.7%

Sample Biographical Information Blank Items

• During high school, how many times did you make the honor roll?

• How much freedom or independence did your parents allow you in grade school?

• How important did your favorite high school teachers stress discipline in the

classroom?

• How many times did you change schools before you were sixteen years old?

• Compared to other people in high school, how many friends did you have?

• How old were you when you spent your first week (or more) away from your

parents?

• How bothered are you if you a job is left undone?

• How often do you read craft and mechanics magazines?

• How quickly do you normally work?

• How well do you feel you can understand the feelings of others?

• How well do you tolerate performing routine tasks?

Honesty Testing

Types of Questions:

1) Frequency and extent of theft (e.g., What percentage of people take more than $1.00 per week from their employer?)

2) Punitiveness toward theft (e.g., should a person be fired if caught stealing $5.00?)

3) Thoughts about theft (e.g., Have you ever thought about taking company merchandise without actually taking any?)

4) Perceived ease of theft (e.g., How easy would it be for a dishonest person to steal from an employer?)

5) Likelihood of detection (e.g., What percent of employees thieves are ever caught?

Validity Issues:

a) Correlations with polygraph resultsb) Future behavior (e.g., # days with cash shortage, discharges)c) Admissions of past theftd) Shrinkage reductione) Contrasted groups (e.g., scores by criminals vs. general population scores)

Employment Interview

• Frequently used to make selection decisions (over 90%)

• Social exchange (interpersonal) process

• Search for information

COMMON PROBLEMS WITH THE “TRADITIONAL” INTERVIEW

•Variety of Interviewer Biases * 1st Impressions * Expectancy Effect * Contrast Effect * Stereotype Matching

•Different Questions Asked to Applicants (Lack of standardization)

•Disagreement on the Desirability of Interview Responses

•Little Formal Interviewer Training

•Subjective (or no) Scoring System

•Interview Conducted and Scored by One Person

•Poor Reliability, Validity, and Job Relevancy (Open to Legal Challenge

Overview of Situational Interview Process

1) Perform a Job Analysis Using the Critical Incident Technique

2) Place Critical Incidents into Relevant Job Dimensions (e.g., Safety, Responsibility, Interpersonal Skills)

3) Reword Critical Incidents Into Question Format

Incident: The employee was married for a year and a half and used any excuse to stay home. One day the employee’s children got colds and no one was around to care for them. So, the employee didn’t show up for work and didn’t phone in.

Question: Your two teenage children are home in bed sick with colds. No friends or relatives are available to watch them. Your shift starts in two hours. What would you do in this situation?

4) Decide on the desirability of responses [Think of how good, average, and mediocre workers would have answered such a question]

_____ _____ _____ _____ _____ 1 3 5 Stay home Phone in & Go in, they just explain the problem have colds

5) Conduct interviews in groups of two or more. Each interviewer scores applicant independently. A single score is given after group discussion

Verbal Reasoning

..… is to water as eat is to …..

A. continue ----- driveB. foot ----- enemyC. drink ----- foodD. girl ----- industryE. drink ----- enemy

..... is to night as breakfast is to …..

A. supper ----- cornerB. gentle ----- morningC. door ----- cornerD. flow ----- enjoyE. supper ----- morning

….. is to one as second is to …..

A. two ----- middle B. first ----- fire C. queen ----- hill D. first ----- two E. rain ----- fire

Differential Aptitude Test (DAT)

Numerical Ability

Add A. 830 B. 1520 C. 16 D. 26 N. none of these

Add A. 1413 B. 1612 C. 25 D. 59 N. none of these

Differential Aptitude Test (DAT)

Abstract Reasoning

PROBLEM FIGURES

ANSWER FIGURES

A B C D E

Differential Aptitude Test (DAT)

Abstract Reasoning (cont.)

PROBLEM FIGURES

ANSWER FIGURES

A B C D E

Differential Aptitude Test (DAT)

Mechanical Reasoning

Which weighs more?(If equal, mark C.)

A B

Differential Aptitude Test (DAT)

Space Relations

A B C D

Differential Aptitude Test (DAT)

SpellingA) man B) Gurl C) Catt D) dog

Language Usage

1) I just / left / my friends / house.

A B C D

2) Ain’t we / going to / the office / next week? A B C D

3) I went / to a ball / game with / Jane.

A B C D

Differential Aptitude Test (DAT)

Clerical Speed and Accuracy

V. AB AC AD AE AF

W. aA aB BA Ba Bb

X. A7 7A B7 7B AB

Y. Aa Ba bA BA bB

Z. 3A 3B 33 B3 BB

Differential Aptitude Test (DAT)

Work Sample Tests Performing (performing a piece of the job

Job Candidates Assessors (evaluators)

Sample Individual Exercises

Interview Simulation

Scheduling Exercise

In-Basket

Sample Group Exercises

Leaderless Group Discussion

Business Game

Exercises

Observe and record candidates

behavior in exercises

Sample dimensions

Decisiveness, Initiative, Judgment, Problem analysis,

Oral communication, Planning, Leadership

Rate candidates on performance dimensions

Discuss and give final ratings

to candidates

-- Assessment Center Process --

Hogan Personality InventoryPrimary Scales ---

Adjustment (confidence, self-esteem, composure under stressful situations)

Ambition (competitive, possessing initiative, potential for leadership)

Sociability (extraverted, friendly, enjoys social interactions)

Likeability (warm, charming, capable of maintaining relationships)

Prudence (responsible, possessing self-control, conscientious)

Intellectance (imaginative, curious, creative)

School Success (achievement orientation, keeps current of business and technical developments)

Self-Report Inventories

Hogan Personality Inventory (cont.)

Occupational Scales ---

Service Orientation (attentive, pleasant, courteous to others such as customers and clients)

Stress Tolerance (ability to handle stress)

Reliability (integrity, good organizational citizen)

Clerical Potential (able to follow directions, attentive to detail, clear communicator)

Sales Potential (energetic, ability to interact socially, able to deal with client issues/problems)

Managerial Potential (leadership ability, good at organizing, capability to make decisions)

Myers-Briggs Type Indicator

Introversion Extraversion(Internally motivated; enjoys spending time (Action-oriented; motivated enjoys alone; prefers one-to-one communication) by outside world and social

interactions)

Sensing Intuitive(Desires concrete information; practical; (Imaginative, creative factual and detailed in orientation) improvises comfortable

with ambiguity; focuses on contexts and connections

Thinking Feeling

Judging Perceiving

Self-Report Inventories (cont.)

Rorschach Inkblot TestTen cards which bilateral and symmetrical inkblots

Scoring ---

Location: the part of the blot used (e.g., use of the whole blot, common or unusual detail)

Determinants: form, color, shading, and movement)

Content (human figures, animal figures, anatomical diagrams, inanimate objects

TAT --- 31 pictures that depict a variety of social and interpersonal situations. Participants are requested to write or tell a story about each picture to the examiner (e.g., what happened, what Ten pictures are gender-specific; the others can be used with either sex.

Use: To uncover internal conflicts, dominant drives, interests, and motives. Specific motives include the need for achievement, need for power, the need for intimacy, and problem-solving abilities.

Thematic Apperception Test

Ability Tests

Types of Selection Tests (cont).

Sensory (e.g., hearing, vision)

Motor (e.g., dexterity, strength, agility)

Cognitive (e.g., Intelligence)