what mathematics do people really use in...

1

WHAT MATHEMATICS DO PEOPLE REALLY USE IN THE WORKPLACE?Merrilea J. Mayo, Lead Strategist

New Options, New Mexico

[email protected]

June 3, 2012

A W.K. Kellogg Foundation funded projectNewOptions NEW MEXICO

2

There Are Tests for Workplace Mathematics, Some Quite Good.

ACT has profiled over 16,000 jobs and designed this test to match the math needed in over 90% of those jobs.

3

Cognitive Skills Tests in General Are Much Better at Predicting Who Will Be “Successful on the Job” Than a Degree Does

Ability Composite = measurement of 3 cognitive skills, strongest predictor of on-the-job performance

WorkKeys Math test = 1 cognitive skill, little less strong predictor

NewOptions NEW MEXICO MEASURING WORK READINESS

4

Another Way of Looking at the Numbers

VARIATION IN INDIVIDUALS’ ON-THE-JOB PERFORMANCE

IS SUCCESSFULLY EXPLAINED BY:

These numbers are the prior validity numbers, squared.


Unknown 99%

Level of Education 1%

Unknown75%

Combined (3) Cognitive Ability

Tests, 25%

Math Skill Test Alone, 16%

5

Skills (Ability Composite) Tests Predict Performance on Complex Jobs Better than Simple Jobs

JOB COMPLEXITY

(as determined from DOL’s “data” dimension in its Dictionary

of Occupational Titles)

HOW WELL SKILLS TESTING PREDICTS PERFORMANCE

(RAW VALIDITY)

1 = perfect prediction of performance0 = zero ability to predict performance

High Complexity .58

Medium Complexity .51

Low Complexity .40

Unskilled .23

References: J.E. Hunter and R.F. Hunter, “Validity and Utility of Alternate Predictors of Job Performance,” Psych Bull 96, 72-981984


6

Literally over 1,000 studies, over 40 years and many job types.

Literature Supporting Validity of .3 to .6 for Cognitive Skill Tests’ Prediction of Job Performance

• M.M. Olea and M.J. Ree, J. Appl. Psych 79, 845-851 (1994).

• M.H. Ree, J.A. Earles and M. Teachout, J. Appl. Psych 79, 518-524 (1994).

• K. Brown, H. Le and F.L. Schmidt, Intl. J. Selection and Assessment 14, 87-100

(2006).

• Schmidt, Ones, and Hunter, Ann. Review Psych 43, 627-70 (1992).

• J.E. Hunter and R.F. Hunter, Psych Bull 96, 72-98 (1984) Also: a full decade of

papers by Hunter from 1980-1990, covering an enormous number of datasets.

• K. Pearlman, F.L. Schmidt and J.E. Hunter, J. Appl. Psych 65, 373-406 (1980).

• F.L. Schmidt, J.E. Hunter, and J.R. Caplan, J. Appl. Psych 66, 261-273 (1981).

• L.C. Northrup, Validity Generalization Results for Apprentice and Helper-Trainer

Positions. Washington, DC: U.S. Office of Personnel Management, Office of

Staffing Policy (1986).

• J.A. Hartigan and A.K. Wigdor, Fairness in Employment Testing: Validity

Generalization, Minority Issues, and the General Aptitude Test Battery.

Washington, DC: National Academy of Sciences (1989).

• E.E. Ghiselli, The Validity of Occupational Aptitude Tests. New York: Wiley, 1966.

(1966), Also: E.E. Ghiselli, Personnel Psych. 26 461-477. (1973).

• J.F. Salgado, H, Anderson, S. Moscoso, C. Berta and F. De Fruyt, Personnel Psych

56, 573-605 (2003).

• H.R. Hirsch, L.C. Norhtrup, and F.L. Schmidt, Personnel Psych 39, 399-420 (1986).

• M.H. Trattner, The Vlidity of Aptitude and Ability Tests for Semiprofessional

Occupations Using the Schmidt-Hnter Interactive Validity Generalization

Procedures. Washington, DC: U.S. Office of Personnel Management, Office of

Staffing Policy (1988).


7

EmployerPositions Filled Using WorkKeys

Outcomes(compared to prior practices)

Subaru of Indiana Automotive (Lafayette,

IN)

Automotive assembly workers

25% reduction in turnover.

Covidian Health Care (Macon, GA)

Manufacturing production

Per-person hiring cost reduced from $2,300 to $600 Time-to-hire reduced from 45 days to 17 days Time-to-full-employee efficiency reduced from 2 years to 6 months.

Steelscape(Calama, WA; Richmond & Rancho Cucamonga, CA)

Self-directed manufacturing teams

<4% turnover overall and a “Best Practice” award from the American Psychological Association for its hiring process.

Bradner Village, a retirement community (Marion, IN)

Nurses, nursing assistants, dietary staff, etc.

37% reduction in turnover.

Inova Health System(Falls Church, VA)

Customer assistants & clinical technicians

73% reduction in turnover; $1.8M cost savings.

911 call center(Lexington county, SC)

Call center operators50% reduction in turnover; skills-related turnover went to 0.

Real-World Results, When Employers Hire by Cognitive Skills Tests


8

So We Have A Test of Workplace Math that Accurately Predicts Workplace Performance.

How Is It Different from “School Math”?


9

Content is Different.

NOT INCLUDED

• Trigonometry, Pre-Calculus, Calculus. Algebra is not explicitly included

but is useful for problems involving more complex order-of-operations.

INCLUDED

• Addition, subtraction, multiplication, division (these operations for

negative numbers as well), fractions, decimals, percentages, mixed

numbers, unit conversions, averages & other simple statistics, ratios,

rates, order of operations, perimeters, areas, volumes

All Problems Defined Against Workplace Standards: Math through 6th grade only.


10

Context is Different.

• All problems are “word problems”

• All problems are highly plausible “on-the-job” situations

• None of the problems has explicit math

– No “find the common denominator.”

– No written formulas you are told to rearrange or solve.

– YOU must interpret the English in terms of math and YOU

must choose the correct math tools to solve the problem.


11

Scoring Scheme is Different

• School Math View: Problem A is harder than Problem B because A is

trigonometry (taught in 9th grade) and B is arithmetic (taught in

2nd grade)

• Workplace Math View: Problem A and Problem B are the same.

Both require simple memorization of a fact. There is no way to

solve either problem using reasoning or deduction. Thus, both

questions measure math skill at a sub-literate level.

30°

A. What is the sin of 30°?

2+2=?

B. What is the sum of 2+2?

12

• More and more extraneous information to sort through.

• More and more rearranging of information required to

get to the answer.

• More and more chained steps. Sequencing is important.

In sum: Workplace Math is “critical thinking,” as applied to math.


In Tests of Workplace Math Skill, Higher Level Questions Have the Following Features:

13

Applied Mathematics Level 3(Qualified for Entry Level Job)


14

Applied Mathematics Level 5(Qualified for Skilled Labor or College)


15

Applied Mathematics Level 7(Qualified for Professional Level Work)


16

If We Measure the U.S. Population Against U.S. Jobs in Terms of Workplace Math Requirements, What Do We Find?


17

WorkKeys APPLIED MATH

0

20

40

60

80

100

0

20

40

60

80

100

3 4 5 6 7

% U.S. Population W

ho

Can Do the Job

(is a

t re

quired s

kill le

vel or

hig

her)

% U.S. Jobs

Skill Level

(N=798,589)(N=4,298)

(8% job-illiterate math)

Data from ACT, http://act.org/workkeys/charts/reading.html, http://act.org/workkeys/charts/math.html


}

(Un)fortunately for the US, most US jobs don’t require much math.

18

However, some important jobs do. (list of occupations requiring AM Level 7)


19

But Why Can Only 6% of the US Population Score a Level 7 in 6th Grade Math?

• The ability to code a “real situation” into math, i.e., to

apply math, is almost never taught and completely

missing, not only in K-12 but also the graduate level. This

may also explain the US’ poor PISA results.

NewOptions NEW MEXICO Month Day, Year

20

y = 0.8826x + 65.719R² = 0.1049

60

65

70

75

80

85

90

95

7 8 9 10 11 12 13 14 15 16 17

WorkKeysScale Scores: (65-90)

Years of Education

The ability to Apply Math Increases Agonizingly Slowly Through School

Years of EducationN

Size

7th Grade 10

8th Grade 33

9th Grade 72

10th Grade 122

11th Grade 252

12th Grade/HS Graduate/GED 974

13 1st year Postsecondary 92

14 2nd year Postsecondary 128

15 3rd year Postsecondary 51

16 4th year Postsecondary 41

17 5th year or higher 34

TOTAL 1,809

>3

3

4

5

6

7

WorkKeys Skill Levels

Correlation= .32

Data from NMDWS; analysis by Tom Kilijanek


21

Targeted curricula can do in one year, what it takes school 6-10 years to accomplish.

Applied MathGain

Locating InfoGain

Reading for InfoGain

Required Keytrain(California data,Kilijanek analysis

5.8/yr(N=132; p<0.001)

5.54/yr(N=218;p<0.001)

8.2/yr(N=63;p<0.001)

Optional Keytrain(Dona Ana Community College data)

5.6/yr(N=38; p<0.1)

3.6/yr(N=38; not sig)

2.8/yr(N=38; not sig)

School (N≈1800, New Mexico)

0.88/yr 0.74/yr 0.85/yr

22

0

20

40

60

80

100

0

20

40

60

80

100

3 4 5 6

% U.S. Population W

ho Can

Do the Job

(is a

t re

quired s

kill le

vel or

hig

her)

% U.S. Jobs

Skill Level: Locating Information

(N=660,044)(N=4,910)

8% illiterate in this skill

Understanding Charts & DiagramsIs Actually a Bigger Problem than Applied Math

Data from ACT, http://act.org/workkeys/charts/locate.html


There is not enough “excess” high skill population for employers to demand this skill at high levels PLUS any other criterion (e.g., individual lives in their state, has a particular degree, is a U.S. citizen, has prior experience).

23

National Deficiency in Locating Information is a Major Contributor to the “STEM crisis.”


Occupations Requiring Locating Information Level 6.

24

The “Workplace Math” View of the World is So Different

WORKPLACE SKILLS VIEW

• You need to be able to apply math,

but only at the 6th grade level.* *some exceptions apply

• Pre- and post-testing of even

college algebra class show zero

gains in applied math skill.

• School degrees predict 1-10% of

work performance variation.

CONVENTIONAL WISDOM

• You need a lot of math to get a

good job.

• Taking advanced math courses

gives you the ability to apply

math.

• School degrees are a good first

cut to predict who will perform

on the job.


25

The Inability to Recode Real World Situations into Math Persists through Graduate School

26

Proof: Employers are Desperate to

Hire Quants

Junior Quantitative Analyst, PHD Machine Learning or Signal Processing – Investment Bank NYC $250k

PHD from a well-regarded university, in Physics, Computer Science, Machine Learning, Signal Processing, Computer Vision, Statistics, Econometrics, Operations Research

PhD Quant Analyst (High performance Computing) – Fixed Income Research NYC $250k

PhD in a Quantitative/Scientific subject. PhD from a top tier University.

Quant Developer/ Algo Developer- Algorithmic Trading Firm- Chicago $175-

250K To qualify for this position, you will have a Bachelors, Masters or PhD in Computer

Science etc from a red brick University with a high GPA.

From one week’s job postings in Quantjobs.com

27

But They Consider the Graduates

from the 110 M.S Programs In this Field

Unemployable

28

How Can This Be?

“All over the world, it has become fashionable for Universities and Colleges to offer Masters degree programs in quantitative finance or financial engineering (FE), a code word meaning the solution of the Black-Scholes option pricing differential equation in as many ways as possible.

“Unless the [academic] field re-invents itself pronto and starts becoming relevant to what people actually do out there, graduates with newly minted financial engineering degrees hoping to see a decent return on their own or their parents’ sizable investment will continue to be sorely disappointed by their actual career prospects, and will keep wondering where in God’s name they went wrong.”

-Sylvain Raynes, Founding Principal of R&R Consulting, a structured credit metrics consultancy

29

We need to teach students to see the

world in terms of math, or they will

be unemployable.


30

In New Mexico, “degree-based hiring” is being replaced by “skill-based hiring.”

Your score on the applied math test determines whether you get a job.

NewOptions NEW MEXICOMEASURING WORK

READINESS

I need someone who is a Level 5 in Listening,a Level 4 in Applied Math

That’s me!!! I have those scores!

31

Skills-Based Job Board


32

Skills-Based Job Board


33

NM Employer Benefits Similar to Those Seen Nationally

• Turnover goes down to <2%.

–Buckman Regional Water Treatment

Plant: 0% after 1 year.

• Time to full employee proficiency is

typically halved (less training time).

• Time and Cost to Hire typically halved

– Farmington Public Library: Hiring

costs went from $2,500 to $750,

5 days to 2.


34

We Took Jobless, High-Poverty, At-Risk Youth in New Mexico, Selected Only the Ones with High Skill Scores, and Gave them Directly to Employers

“He’s fantastic! We are so pleased with his work,

and we would not have had him in our employment

if it wasn’t for you and New Options.”

--Smart Motors

“We were unsure whether New Options NM

would be able to send us the right person amazing

pleasure for us to work with. We have trained her

on both Quickbooks and as a telephone receptionist.

Madai did not like math, but now understands that

math can actually be fun if it is in using Quickbooks.

We think this is a good opportunity for our company

to mentor a youth, and to understand there is a

different talent pool that exists to tap into.”

--Santa Fe Title Company

“Candance has just done a

wonderful job. We are very pleased

and would love to have her as a

full-time employee if a position

existed. Dina and she are working

on accessing positions that are

available in our district and we will

help her apply. She is well

prepared, knowledgeable and

through her training with you,

it is obvious that she is well

prepared to enter the work force.”

--Santa Fe Public SchoolsTraining? We didn’t train anyone! They were already this good.


35

More Employer Quotes…

“We enjoy her very much. She was a great hire

for the summer. We actually offered her a

permanent job, but she decided she was going

to attend UNM this fall. I do like WorkKeys

because it helped determine who was eligible

for the job and who could handle the work, so

that is a big plus for WorkKeys. To be honest,

we would’ve found someone else through our

ads, but I’m glad we used WorkKeys.”

--Vogue College of Cosmetology

“Danielle is very qualified for the job, is kind,

sweet and does what she is told in a timely

manner. We found her through the New

options NM WorkKeys project. She has

performed at all tasks given to her and we

would very much like a way to keep her on

when she goes back to high school in the fall.”

--First Tee of Santa Fe

“Without the program we would not

have had any contact with our seven

candidates for the job, and Marissa

was one of them. Marissa was the only

one who came in, asked questions,

and seemed generally interested in

the greenhouse and what we do. She

was the only one out of the seven that

was engaged during the interview

process. She has done everything she

has been asked to do and keeps the

greenhouse looking good. She shows

initiative in her work; getting started

and working with-out needing help or

supervision. She is very good and who

you hope to get in the workplace.”

--New Earth Orchid


what mathematics do people really use in...

Documents