an examination of the program for the international assessment of adult competencies (piaac)...

An Examination of the Program for the International Assessment of Adult

Competencies (PIAAC) Findings in the United States

National Council for Workforce EducationPittsburgh, PA

October 30, 2014

Phyllis Cummins and Ryan Walker

PIAAC Background

Literacy, Numeracy, Problem Solving Skills

Overview of research process

Results

Discussion

Questions

Presentation Overview

• Coordinated by the OECD• International large-scale assessment administered in

2011-12 in 23 countries• Third in a series of assessments: IALS between

1994-1998 and ALL between 2003-2008• 16- to 65-year-olds, non-institutionalized, residing in

the country• Laptop computer or paper-and-pencil• Assessment subjects: Literacy, Numeracy, and

Problem Solving in Technology-Rich Environments • Extensive background questionnaire

What is PIAAC?

Participating Countries

2012 2015

AustraliaAustriaBelgiumCanadaCyprusCzech RepublicDenmarkEstoniaFinlandFranceGermanyIreland

ItalyJapan Korea, Rep ofNetherlandsNorwayPolandSlovak RepublicSpainSwedenUnited Kingdom United States

ChileGreece IndonesiaIsraelLithuaniaNew ZealandSingaporeSloveniaTurkey

Source: Adapted from IES/PIAAC Presentation, Oct. 2013

U.S. Adults Fare Poorly in a Study of SkillsBy RICHARD PÉREZ-PEÑA Published: October 8, 2013

New York Times Article

• Below the international average in all subject areas• Ranked better in literacy than in numeracy or problem

solving in technology-rich environments• Higher percentage at low proficiency levels than

international average• Percentages of top performers similar to or slightly

lower than international average, depending on the subject

• Performance gap between young and older population smaller than the average gap internationally

General Patterns of U.S. Results

U.S. average literacy score (270) lower than the international average (273)


U.S. Average Literacy Score in 2012 not Significantly Different

from 2003, but Lower than in 1994

1994 2003 2012245

250

255

260

265

270

275

280

285

IALS:273*

ALL:268

PIAAC:270

*p < .05. Average score is significantly different from PIAAC.


Only Oldest U.S. Adults Outperformed the International

Average in Literacy 55-6545-5435-4425-3416-24

55-6545-5435-4425-3416-24

PIAA

C in

tern

ation

al a

vera

ge

0 50 100 150 200 250 300

263*266

273*275*

272*

255268

279284

279

*p < .05. U.S. average score is significantly different from PIAAC international average.


Least Educated Adults Below the International Average in Literacy

Graduate or professional degree

Bachelor's degree

Associate's degree

High school credential

Below high school


Bachelor's degree

Associate's degree


Below high school

Uni

ted

Stat

esPI

AAC

inte

rnati

onal

ave

rage

0 50 100 150 200 250 300 350

310

298

283

262*

230*

307

298

285

271

246



Employed Adults in the U.S. Had Lower Average Literacy Scores

than Their Peers Internationally Employed

Unemployed

Out of the labor force

Employed

Unemployed


PIAA

C in

tern

ation

al a

vera

ge

0 50 100 150 200 250 300

274*

260

257

277

265

261



U.S. average numeracy score (253) lower than the international average

(269)


U.S. Average Numeracy Score in 2012 Lower Than

in 2003

2003 2012235

240

245

250

255

260

265

270

275

ALL:262*

PIAAC:253

*p < .05. Average score is significantly different from PIAAC.


U.S. Adults in All Age Groups Below International Average in

Numeracy55-6545-5435-4425-3416-24

55-6545-5435-4425-3416-24

PIAA

C in

tern

ation

al a

vera

ge

0 50 100 150 200 250 300

247*250*

258*260*

249*

253265

275279

271



U.S. Adults at Every Education Level Below the International Average in

NumeracyGraduate or professional degree

Bachelor's degree

Associate's degree


Below high school


Bachelor's degree

Associate's degree


Below high school

Uni

ted

Stat

esPI

AAC

inte

rnati

onal

ave

rage

0 50 100 150 200 250 300 350

302*

287*

267*

243*

204*

308

296

283

268

237



Employed Adults in the U.S. Had Lower Average Numeracy Scores than Their Peers

Internationally

Employed

Unemployed


Employed

Unemployed


PIAA

C in

tern

ation

al a

vera

ge

0 50 100 150 200 250 300

260*

236*

232*

275

256

253



U.S. Average Problem Solving in Technology-Rich Environments Score (277) Lower Than the International

Average (283)


Only Oldest U.S. Adults Outperformed the International Average in Problem Solving

in Technology-Rich Environments55-6545-5435-4425-3416-24

55-6545-5435-4425-3416-24

PIAA

C In

tern

ation

al a

vera

ge

0 50 100 150 200 250 300 350

267*271

279*283*285*

259272

285295295

*p < .05. U.S. average scores are significantly different from PIAAC international average.


Least Educated Adults Below the International Average in Problem

Solving in Technology-Rich Environments

Graduate or professional degreeBachelor's degree

Associate's degreeHigh school credential

Below high school

Graduate or professional degreeBachelor's degree

Associate's degreeHigh school credential

Below high school

PIAA

C in

tern

ation

al a

vera

ge

0 50 100 150 200 250 300 350

301296

282268*

259*

304299

286278

269



Employed Adults in the U.S. Had Lower Average Problem Solving in Technology-Rich Environments

Scores than Their Peers Internationally

Employed

Unemployed


Employed

Unemployed


PIAA

C in

tern

ation

al a

vera

ge

0 50 100 150 200 250 300

279*

271*

272*

285

279

278



Lower overall scores than international average in all subjects

Higher percentages of low performers than internationally

Larger gaps between less advantaged and more advantaged peers in literacy and numeracy, but not in problem solving in technology-rich environments

Relatively lower performance of young adults and those with high school education or less

Relatively higher performance of older adults in literacy and problem solving in technology-rich environments

U.S. PIAAC Findings Summary:


The Current Research Project:

Examination of the Relationship between

Participation in AET and Labor Market Outcomes

Background• Investments in human capital are becoming

increasingly important in a knowledge based global economy

• The U.S. labor force in aging• People are remaining in the labor force at older

ages• Older workers are important for economic growth• Older adults may have a fear of returning to the

classroom and test-taking

PIAAC and AET Participation in the U.S.

U.S. Labor Force Distribution by Age Group

1992 2002 2012 Projected 2022

17.3

22.5 22.619.3

9.011.3

15.9 17.3

2.7 3.15.0

8.2

29.0

36.9

43.6 44.9

45 to 54 55 to 64 65 and older 45 and olderSource: Toossi (2013)

U. S. Male Labor Force Participation by Age Group

45 to 54

55 to 64

65 to 74

90.7

67.0

21.1

88.5

69.2

25.5

86.1

69.9

31.8

85.1

71.0

35.9

2022 (projected) 20122002 1992

Source: Toossi (2013

U. S. Female Labor Force Participation by Age Group

45 to 54

55 to 64

65 to 74

72.6

46.5

12.5

76.0

55.2

16.1

74.7

59.4

22.5

74.9

64.3

28.3

2022 (projected) 2012 2002 1992

Source: Toossi (2013)

• Purpose of research– To identify the relationship between participation in formal and

non-formal adult education and training (AET) programs and labor market outcomes for ages 45 to 65

• Research questions:– Does participation in formal and non-formal AET predict

employment status?– Does participation in formal and non-formal AET predict labor

force participation?– Does participation in formal and non-formal AET predict

income quintile?

Current Project

• Similar to “Lifelong Learning”• Can be formal, non-formal, and informal

– Formal - learning that takes place in education and training institutions and leads to recognized credentials and diplomas

– Non-formal - learning that takes place in educational and training settings, but does not typically lead to a formalized credential

– Informal - learning that takes place in everyday life and is not necessarily intentional

• Job related and non-job related• Our focus was on formal and non-formal education

What is Adult Education and Training?

• Descriptive statistics• Used chi-square tests and logistic regression

techniques • PIAAC used a complex sample design that involved

both sample weights and replication weights

Methodology

Results

Descriptive Statistics:Characteristics of AET Population

45 - 54 55 - 65Age Group 53.1 46.9Male 47.6 47.1Female 52.4 52.9Education

Lower secondary or less 11.4 10.4Upper secondary 40.7 42.8Post-secondary non-tertiary 9.9 8.7Tertiary professional degree 9.7 8.0Tertiary bachelor's degree 16.5 15.8Master/research degree 11.8 14.3

Employed 93.2 93.5Participates in Labor Force 85.9 69.0Participated in AET in prior 12 months Formal and non-formal AET 55.8 50.4 Formal and non-formal AET, job related 47.1 39.1 Non-formal AET 53.7 48.9 Non-formal AET, job related 45.4 38.1

Descriptive Statistics:Reasons for Non-Participation by Age Group

No employer support

Lacked prerequisites

Something unexpected came up

Class offered inconvenient time or place

No time-family demands

Too expensive

Too busy

1.1

4

7.1

32

8.6

29.5

19.6

3.9

1.2

4.1

9.4

22.3

29.6

29.5

2.9

2.9

5.5

10.6

25.4

24.4

28.2

6.2

1.5

9.4

11.6

13.4

22.4

35.5

5.2

2.5

6.2

19.6

12.4

24.2

29.9

55-65 45-54 35-44 25-34 <25

Descriptive Statistics:Reasons for Non-Participation by Education

No employer support





Too expensive

Too busy

3.0

9.4

8.1

12.2

20.8

23.1

23.3

2.6

2.6

8.9

15.1

14.7

28.3

27.8

4.9

1.3

4.4

14.9

19.2

25.0

30.4

Beyond upper secondary Upper Secondary Lower Secondary

Descriptive Statistics: Reasons for Non-Participation by Income Quintile

No employer support





Too expensive

Too busy

2.1

3.3

13.2

18.7

11.4

35.2

16.1

4.1

2.9

3.9

11.9

16.8

35.3

25

3.1

2.1

3.4

13.0

16.3

26.7

35.4

5.4

0.9

4

19

13.9

18.1

38.6

6.3

1.3

2.9

10.7

14.6

12.2

52

Highest Next Highest Middle Second Lowest Lowest

Descriptive Statistics: Reasons for Non-Participation by Sex

No employer support





Too expensive

Too busy

4.4

2.6

5.6

17.1

11.6

23.7

35

3.5

2.1

6.9

12.9

22.5

27.9

24.2

Female Male

Chi-square Test Results: Ages 45 - 54

Non-Participation Group

Participation Group Significance

n (%) n (%) Design BasedF df, rdf

pEmployment Status Unemployed 41 (70.1) 21 (29.9) 24.98 1, 79 <.001

Employed 298 (35.5) 537 (64.5)

Labor Force Participation

Not in the labor force 131 (81.4) 31 (18.6) 104.42 1, 79 <.001

In the labor force 339 (37.8) 558 (62.2)

Income Quintile

Lowest 57 (52.6) 46 (47.4) 15.36 3.64, <.001

Next Lowest 69 (48.0) 68 (52.0) 287.83

Middle 55 (42.0) 87 (58.0) Next to highest 40 (23.2) 126 (76.8)

Highest 32 (16.6) 174 (83.4) Total n (income) 253 (33.7) 501 (66.3)

Chi-square Test Results: Ages 55 to 65

Non-Participation Group

Participation Group Significance

n (%) n (%)Design Based

F df, rdf p

Employment Status

Unemployed 22 (43.8) 27 (56.2) 1.04 1, 44 0.314

Employed 225 (36.0) 430 (64.0)

Labor Force Participation

Not in the labor force 264 (78.9) 75 (21.1) 172.70 1, 44 <.001

In the labor force 247 (35.1) 457 (64.9)

Income Quintiles

Lowest 41 (49.5) 49 (50.5) 4.20 3.67, <.05 Next lowest 30 (36.6) 50 (63.4) 161.35

Middle 45 (40.1) 74 (59.9)

Next to highest 37 (30.0) 97 (70.0)

Highest 33 (22.6) 111 (77.4)Total n (income) 186 (34.2) 381 (65.8)

Regression Results

Model 1Employment

Model 2Labor Force Participation

Model 3Income

Did not participate in FNFAET12 (ref. grp.) - - -

Participated in FNFAET12 0.684*** 1.384*** 0.441***(0.149) (0.112) (0.0840)

Age (all age groups) 0.262***

(0.0629) Ages 18 to 44 (ref. grp.) - Ages 45 to 65 -0.382*** 0.593***

(0.0989) (0.0758)Male (ref. grp.) - - -Female -0.376** -0.984*** -1.027*** (0.141) (0.103) (0.0765)Education Lower secondary or less -0.259 -0.316* -0.686***

(0.221) (0.156) (0.131)Upper secondary (ref. grp.) - - -Post-secondary, non-tertiary -0.0142 0.196 0.211

(0.247) (0.183) (0.136)Tertiary professional degree 0.219 -0.0315 0.714***

(0.265) (0.182) (0.137)Tertiary bachelor's 0.631** 0.0894 1.331***

(0.219) (0.150) (0.109)Master’s/ research degree 0.620* 0.266 2.070***

(0.281) (0.193) (0.129)Observations 3,540 4,325 2,952

• There was a significant relationship between AET and employment status

• There was a significant relationship between AET and labor force participation

• The unemployed and not-in-the labor force groups are less likely to participate in AET

• Could increased AET participation by the unemployed reduce long-term unemployment and early exit from the labor market?

Discussion

• AET participation increases the likelihood of moving up one income quintile

• Lower income quintiles less likely to participate in AET than higher income quintiles

• Females benefit less than males from AET participation

• Less than a high school diploma group benefits less than high school graduates from AET participation

Discussion

• Provide additional low-cost opportunities for AET participation by low-income groups and the unemployed

• Improve outreach programs to the unemployed• Increase supportive services to older adults

returning to the classroom• Encourage participation in both formal and non-

formal training opportunities

Recommendations

• Examine the relationship between literacy, numeracy, and problem solving skills with AET participation

• Examine outcomes for 66 – 74 age group• Further examine characteristics of the

unemployed• Compare PIAAC outcomes with IALS and ALL

for the baby boomer cohort

Future Research

This project was commissioned by American Institutes for Research,

funded through a contract with the National Center for Educational

Statistics (NCES).

Questions?

an examination of the program for the international assessment of adult competencies (piaac)...

Documents