tools for successful selection employment testing: general employment tests lecture.. rowena r....
TRANSCRIPT
Tools for Successful Tools for Successful SelectionSelectionEmployment Testing: General
EMPLOYMENT TESTS LECTURE.. ROWENA R. MANANSALA1
Types of Employment Types of Employment TestsTests1. Ability Tests
◦Mental (Cognitive) Ability◦Psychomotor Ability◦Physical Ability◦Sensory Ability
2. Achievement and Skill Tests◦Job Knowledge◦Performance/Work Sample
Selecting Employment Selecting Employment TestsTests
Which KSAs are relevant to testing?Search sources of tests
◦General Tests: A Comprehensive Reference... Web-based information sources
◦Reviews Mental Measurement Yearbook
Useful article on using the MMY British Psychological Society Test Critiques
Evaluating Test ManualsEvaluating Test ManualsEvidence of reliabilityRelevant evidence of validity
◦Use of appropriate criteria◦Norms, based on appropriate
samplesEvidence of test fairnessPracticality
““Successful” IntelligenceSuccessful” Intelligence
Analytical intelligence◦“Academic” intelligence
Practical Intelligence◦“Common sense” as well as social or
emotional intelligence; “street smarts”Creative Intelligence
Ability to achieve success in life and work; requires a balance among:
Analytical IntelligenceAnalytical Intelligence
Traditional aspects of intelligence measured by standard tests◦ Verbal and numerical ability, memory,
processing speedPredicts job performance in nearly all jobs
◦ More important as jobs become more complex◦ Probably due to:
Broader knowledge base Ability to learn on the job more quickly Ability to problem-solve
Practical IntelligencePractical IntelligenceAbility to deal with everyday life challenges
◦ E.g., computing lowest price/unit when grocery shopping, or interpreting a bus schedule
Social/emotional intelligence◦ Ability to understand people and relationships◦ Impression management skills◦ Ability to persuade others
Adaptability to different social situationsShown to add to predictive power of
analytical intelligence for sales and execsMay be culture-specific
Creative IntelligenceCreative Intelligence
Relies on a base of knowledge + ability to apply it to new situations or problems ◦Ability to generate novel high-quality
ideas◦Ability to recognize opportunities or
define challenges in new ways◦Ability to adjust to new situations
Measuring IntelligenceMeasuring Intelligence
Wide variety of measures of analytical intelligence/general mental ability◦ Wonderlic Personnel Test is most popular
Sternberg Multidimensional Abilities Test measures all 3 intelligences
Sternberg and Wagner’s test of Practical Managerial Intelligence: ◦ ability to write effective memos ◦ ability to motivate people ◦ knowledge of when to delegate ◦ ability to “read” people
Lingering Controversies Lingering Controversies about IQabout IQ
Is intelligence testing fair?Is intelligence—even when
measured more broadly—sufficient?
Do you want “bright” or “well trained” employees?
Fairness and Intelligence Fairness and Intelligence Testing: A DilemmaTesting: A DilemmaCA predicts performance in a
wide variety of jobs◦Measures knowledge◦Measures ability to learn, reason,
and solve new problems◦Few alternative predictors with
comparable utility for entry-level jobs
But are they “fair” to all groups?
True Validities of Predictors For Entry-Level Jobs
Ability Test Battery .53 Interview (structured) .44 Job Tryout .44 Biographical Inventory .37 Reference Checks .26 Personality Inventories ~.23 Experience Ratings .18 Interview (unstructured) .14 Academic Achievement .11 Interest Inventories .10
For Promotions Work Sample .54 Ability Test Battery .53 Peer Ratings .49 Experience Ratings .49 Job Knowledge Test .48 Assessment Center .43
Definitions of Test Definitions of Test “Fairness”“Fairness”
1. Differences between groups’ average test scores◦ ~1 SD difference for African-Americans◦ ~.75 SD for Latinos
But what do these differences mean?
• Groups differ in ability?• Test discriminates against one group?
Definitions of Test Definitions of Test “Fairness”“Fairness”2. Differential Validity: Group
differences in validity of test
Definitions of Test Definitions of Test “Fairness”“Fairness”2. Differential Validity: Group
differences in validity of test
White
Black
Test Scores
JobPerf
Definitions of Test Definitions of Test “Fairness”“Fairness”3. Test Bias:
Group differences in predicted job performance with same cutoff score
Definitions of Test Definitions of Test “Fairness”“Fairness”3. Test Bias: Group differences in
predicted job performance with same cutoff score
Black
Test Scores
JobPerf
White
Definitions of Test Definitions of Test “Fairness”“Fairness”3. Test Bias: Group differences in
predicted job performance with same cutoff score
Black
Test Scores
JobPerf
White
Typical CA Test SituationTypical CA Test SituationAdverse ImpactNo differential validityNo test bias
White
Black
Test Scores
JobPerf
Typical CA Test SituationTypical CA Test SituationAdverse ImpactNo differential validityNo test bias
White
Black
Test Scores
JobPerf
Minimizing Adverse Minimizing Adverse ImpactImpactRecruit high quality applicantsUse tests only when appropriateUse tests as one part of processUse to eliminate unqualified
(rather than to differentiate among most qualified)
Be sure test is valid (job-related)!!
Lingering Controversies Lingering Controversies about IQabout IQ
Is intelligence testing fair?Is intelligence—even when
measured more broadly—sufficient?
Do you want “bright” or “well trained” employees?
Why Intelligent People FailWhy Intelligent People FailLack of motivation Lack of impulse control Lack of perseverance Fear of failure Procrastination Inability to delay gratification Too little/too much self-confidence
Source: Sternberg (1986)
Lingering Controversies Lingering Controversies about IQabout IQ
Is intelligence testing fair?Is intelligence—even when
measured more broadly—sufficient?
Do you want “bright” or “well trained” employees?
Intelligence or Training?Intelligence or Training?
Is being smart a pre-requisite for learning?Is being smart sufficient for learning?
InvasivenessInvasivenessEmployer right to know vs. right to
privacyOver-emphasis on individual itemsReducing invasiveness:
◦Use only for job-related traits◦Use employment (not clinical) measures◦Use as part of overall system◦Explain purpose as matching not
screening
Extreme Items from MMPI
I have never indulged in any unusual sex practices.
I have never been in trouble because of my sex behavior.
My soul sometimes leaves my body.
Evil spirits possess me at times.
I sometimes feel like I’m going to pieces.
Often I feel as if there were a tight band around my head.
I have had no difficulty starting or holding my bowel movements.
Sample Items Modeled After Those on NEO-PI
I am generally not a worrier.
I would rather cooperate than compete.
I don’t mind bragging about what I’ve accomplished.
I’m pretty set in my ways of doing things.
I try to keep all my thoughts realistic and to avoid flights of fancy.
I have no sympathy for “street people.”
I prefer jobs that allow me to work alone without being interrupted by others.
I am a person who always gets the job done.
Is Personality Job-Is Personality Job-Relevant?Relevant?The “Big Five” Personality Traits
◦Conscientiousness◦Emotional Stability◦Extraversion◦Agreeableness◦<Openness to Experience>
Others may apply in particular jobs ◦e.g., customer service, safety,
attention to detail
Do Personality Tests Do Personality Tests Predict?Predict?Is the situation weak or powerful?Is the measure valid?Validities typically in low .20s
(when based on job analysis)
Faking & Personality TestsFaking & Personality TestsApplicants often distort their
answers to look good◦Some applicants more than others◦This can affect who is hired
Reducing faking:◦Use faking (lie) scales ◦Reduce motivation to fake◦Use less transparent questions
Social Desirability Scale Items
I sometimes drive faster than the speed limit.
My table manners are not quite as good at home as when I’m out in company.
I gossip a little at times.
Sometimes I vote for people I know very little about.
I do not like everyone I know.
I sometimes tell lies if I have to.
Fairness of Personality Fairness of Personality TestsTestsLittle evidence of racial
discriminationMen and women differ on many
scales◦but no evidence of “discrimination”
Watch out for adverse impact due to:◦religious beliefs◦disability
Tools for Successful Tools for Successful SelectionSelectionAssessing Deviance:Drug TestingIntegrity (Honesty) Testing
General RationaleGeneral RationaleEmployee deviance is a costly
problem◦$6 to 200 billion annual cost ◦33-75% of employees
Screening out Deviant Screening out Deviant EmployeesEmployeesSome are based on past behavior
◦analysis of drug metabolites in urine; past theft; reference and background checks
Others focus on likely future behavior◦attitudes toward theft, drug use;
personalityOnly a few focus on current behavior
◦impairment testing
Physiological Drug TestsPhysiological Drug TestsUrinalysis
◦Quick screening◦Confirmatory: Gas
Chromatography/Mass Spectrometry (GC/MS)
Hair analysisBlood analysisBreath analysis
What’s Measured (& for how What’s Measured (& for how long?)long?)
The “NIDA 5”:◦Cannabis◦Cocaine◦Amphetamines◦PCP◦Opiates
Other Forms of Drug TestsOther Forms of Drug TestsPaper-and-pencil integrity testsImpairment testing
◦based on person’s own “baseline” performance on a computer task
Types of Drug Testing Types of Drug Testing ProgramsProgramsPre-employment“For cause” testing
◦mandatory post-accident◦based on supervisor’s observations
Fitness for duty ◦daily (impairment testing)◦post-rehabilitation
Random
Types of Honesty/Integrity Types of Honesty/Integrity TestsTests“Clear Purpose” tests“Disguised Purpose” tests
◦Attitudes about deviance◦Personality
Polygraph/Voice Stress Analyzer
Clear Purpose TestClear Purpose Test - Honesty/Morality Items- Honesty/Morality Items
If you received an extra $1.00 in change when you purchased groceries, would you keep it?
If you saw a co-worker stealing, would you turn her in?
How many people have you known who have stolen valuables at work?
Most people cheat on their income tax.
Clear Purpose Test:Clear Purpose Test: - Drug-Related Items - Drug-Related ItemsI have never been a heavy
drinker. On occasion I have used
alcohol/drugs excessively If I thought I could get away with
it, I'd probably try cocaineHave you ever missed work
because you were too high/drunk?
Types of Honesty/Integrity Types of Honesty/Integrity TestsTests“Clear Purpose” tests“Disguised Purpose” tests
◦Attitudes about deviance◦Personality
Polygraph/Voice Stress Analyzer
Disguised Purpose TestDisguised Purpose Test - - Drug Attitude ItemsDrug Attitude ItemsA drug addict who steals to
support his habit should get the same penalty as a person who steals for profit.
Employers expect a certain amount of drug and alcohol use among their workers.
How many people have you known who drink or do drugs while working?
Disguised Purpose Test:Disguised Purpose Test: - - Personality ItemsPersonality ItemsI enjoy making people feel better.I don't care if others like the
things I do.I shouldn't do many of the things
I do.I have never been in trouble with
the law.Sometimes I enjoy going against
the rules.
Drug & Integrity Testing Drug & Integrity Testing IssuesIssuesLegalityAccuracy/Job relevanceFaking (integrity tests)Consequences of failingNegative reactions (invasion of
privacy/fairness)
Integrity Testing: A CautionIntegrity Testing: A CautionIntegrity tests assume cause of
deviance is bad employeesBut the situation may also be a
factor◦role models, group norms may “push”
people towards deviance ◦dissatisfaction may “pull” towards
devianceIntegrity tests don’t replace good
management
Tools for Successful Tools for Successful SelectionSelectionEmployment Testing: Performance Tests & Work Samples
Signs vs SamplesSigns vs SamplesSigns: indirect indicators of
performance potentialSamples: direct indicators of
performance potential
Sign-Sample ContinuumSign-Sample ContinuumProbationary employmentApprenticeshipsInternshipsMock-upsAssessment centersRole-playing interviews
Paper/pencil tests
Sample
Sign
Pros & ConsPros & Cons
High validityLow adverse impactHigh face validity
Advantages
• More time consuming• Direct costs
Disadvantages
Step 1: Define Step 1: Define Performance Performance Which performance dimensions?
◦Refer to Performance Attributes Matrix
◦Don’t focus only on easily simulated dimensions
Performance Attributes Performance Attributes MatrixMatrix
Attribute
App.
Blank Interview
Reference
Checks
Work
Sample
Knowledge
Test
Personality
Inventory
Ability to workaccurately
X X X X
Ability to workwell with customers
X X X X
Work-specificknowledge
X X X
Computer-operating skill
X X
Conscientiousness X X X
Step 1: Define Step 1: Define Performance Performance Which performance dimensions?
◦Refer to Performance Attributes Matrix
◦Don’t focus only on easily simulated dimensions
Determine reasonable standards for the test/sample
What type of validation would you use?
Step 2: Creating SampleStep 2: Creating SampleKey principles
◦Specificity◦Comprehensiveness (vs Narrowness)◦Fidelity◦Practicality
Use Performance Attributes Matrix to:◦Ensure all KSAs are included in plan◦Weight all KSAs appropriately
Example: Example: Marketing SpecialistMarketing SpecialistPerforming market analysis
◦Have candidates perform a market analysis for a particular product/market
Developing appropriate marketing materials◦Critique current materials
Developing a marketing plan◦Develop part of a plan