future learning skills in a global context: from … · « 21st century skills » ... domains or...

FUTURE LEARNING SKILLS IN A GLOBAL

CONTEXT: FROM DISCOURSE TO

PRACTICE

Stéphan Vincent-Lancrin, Ph.D.

Senior Analyst, Project Leader and Deputy Head of Division

Centre for Educational Research and Innovation,Directorate for Education and Skills

Helsinki, May 9th 2017

Skills and education for innovation« 21st Century Skills »

Innovation

Skills

Educationand

training

skills for innovation

Skills that tertiary-educated professionals

report as very important in their job

Percentage of employees reporting the following skills as very important in their job

Source: Avvisati, Jacotin and Vincent-Lancrin (2014), based on REFLEX and HEGESCO data

22.7

30.6

40.2

40.3

40.4

40.4

40.5

41.8

46.5

48.0

50.0

53.4

54.2

55.0

56.5

56.9

58.6

60.8

61.7

0 10 20 30 40 50 60 70

knowledge of other fieldswrite and speak a foreign language

assert your authoritynegociate

alertness to opportunitiespresent ideas in audience

willingness to question ideasmobilize capacities of others

analytical thinkingcome with news ideas/solutions

write reports or documentsacquire new knowledge

master of your own fieldcoordinate activities

use computers and internetmake your meaning clear

work productively with othersperform under pressure

use time efficiently

Critical skills for the most innovative jobs

(according to tertiary-educated workers)

1,83

2,05

2,08

2,09

2,15

2,19

2,19

2,20

2,35

2,36

2,36

2,42

2,51

2,58

2,60

2,71

2,81

3,00

3,90

1,00

assert your authorityknowledge of other fields

negociateperform under pressure

use time efficientlywork productively with others

use computers and internetwrite and speak a foreign language

write reports or documentsmaster of your own field

make your meaning clearmobilize capacities of others

acquire new knowledgecoordinate activities

analytical thinkingalertness to opportunitiespresent ideas in audience

willingness to question ideascome with news ideas/solutions

Likelihood (odds ratios) of reporting the following skills: people in the most innovative jobs vs. least innovative jobs

Source: Avvisati, Jacotin and Vincent-Lancrin (2014), based on REFLEX and HEGESCO data

What individual skills should

education systems foster?

Technical skills (know-what and know-

how)

Creative and Critical Thinking

skills(Critical thinking,

observation, curiosity, ability to make connections,

imagination,...)

Social and Behavioural skills (Self-confidence, energy,

perseverance, passion, leadership, collaboration,

communication)

Some comments on these skill

categories

• They are domain-specific

– Skills are generally domain-specific: one is creative in a field, one knows how to behave/communicate in a specific context, one has problem-solving skills in a field, one has content knowledge in a field

• They can become « domain-generic »

– A skills becomes « domain-generic » when one has gained it in a number of domains or settings, so that it becomes a « habit of mind » (a disposition or a stabilised skill) that one can apply to new fields

• They overlap and may reinforceeach other

But

• They are different and cannotbe reduced to a single skill (or measure)

Technical skills

Creative and Critical Thinking

skills

Social and Behavioural skills

education for innovation

What the hell do we mean?

• « 21st century » skills are acknowledged and competence-based curricula are in place in virtually all OECD countries

• Teachers do not disagree withthem, but they don’t know whatthey actually mean in practice

• Curricula acknowledge them but remain at odds with them in mostcountries

• What does it mean to develop the creativity and critical thinking?

Innovation

Skills

Educationand

training

1. Can we articulate a common international language?

2. Can we develop an exemplary pedagogical toolkit to teach and assess creativity and critical thinking as part of countries’ (current) curriculum?

3. Can we identify a development (progression) scale for theseskills?

4. What are the key aspects of context that matter for theirdevelopment?

5. What are the effects of using the developed pedagogical toolkiton pedagogies, beliefs, social and behavioural skills, and standardised measures of creativity and academic achievement?

Ongoing project on fostering and assessing

students’ creativity and critical thinking

• Need for a common language, social representation and guidance about what some desired skills actually mean

• Skills that are not assessed are not taught consistently, but teachers also need to teach what they assess

• There is generally little space for students to develop and demonstrate creativity and critical thinking as part of theirusual disciplinary learning

• Start a process of change: pilot, prototype and developpedagogical resources as a proof of concept for otherteachers – before validation and possibly scale up (second phase)

Theory of action

Starting point: 5 creative habits of mind

(21st century skills)

Source: Lucas, Claxton and Spencer (2013)

• Intervention: Use of a (new) common rubric on creativity and critical thinking to develop pedagogicalactivities, assessments and define proficiency levels

• Domains of intervention (2 out of 3):– STEM (science and/or maths)

– Arts (music and visual arts)

– Interdisciplinary

• Minimum sample for intervention and for control group:– ≥ 5+5 schools (9 year olds / 14 year olds)

– ≥ 10+10 teachers (classes)

– ≥ 200+200 students

What we do: basics of the intervention

• Participation in 11 countries overall– Brazil, France (3), India, Hungary, Netherlands,

Russia, Slovakia, Spain, Thailand, UK (Wales), United States (3)

• 1st round (2015-16, 2016) in 9 countries– Brazil, France (3), India, Hungary, Netherlands,

Russia, Slovakia, Thailand, United States (3)

– At least 128 schools, 372 teachers, 9093 students

• 2nd round (2016-17) in 9 countries– Brazil, France, India, Hungary, Russia, Spain,

Thailand, UK (Wales), United States (2)

Participation in primary and secondary

education

Developing a pedagogical toolkit

A. The toolkitA1.

Rubric for assessment

Dimensions

Levels of progression

A2.

Pedagogical activities

Specific for each domain

Designed to test

dimensions and levels of progression

A3.

Set of exercises

Specific for each domain

Designed to prepare for the

assessment

A4.

Portfolio of student work

For each domain

As examples of different skill

levels

Contextual data collection

B1.

Subject-based assessment

Standardised assessment of

academic achievement

(maths and science; visual

arts and music)

B2.

Creativity assessment

Standardised test for

creativity

(EPoC)

(domain-specific)

B3.

Survey questionnaires

School principals

Teachers

Students

B4.

Interviews / focus groups

Teachers

Students

School-based intervention

School year

A A A A

Primary school group - Ages 8-9. STEM, Arts or interdisciplinary domain

Secondary school group - Ages 13-14. STEM, Arts or interdisciplinary domain

A

What factors influence the outcomes?

• Pre-tests and questionnaires at the beginning of the intervention:– Are differences related to baseline in achievement, creativity, to student

beliefs, to pedagogies, to socio-economic bacground, etc.?

• Observations and discussions within the network

B

B

School-based intervention

School year

A A A A

Primary school group - Ages 8-9. STEM, Arts or interdisciplinary domain

Secondary school group - Ages 13-14. STEM, Arts or interdisciplinary domain

A

What effects of the intervention?

• Measures after the intervention:– Post-tests and -questionnaires

– Qualitative observations of pedagogies

– Interviews, ESM, games to test executive functions

• Matched control group (with some kind of intervention as well)

Control group

Control group

B

B

B

B

• Domain-specific creativity tests: – Maths, Science, music, Graphic artistic, social

EPoC (Evaluation of Potential Creativity)

– All (except graphic) are still experimental

• Achievement tests:– Maths and Science: TIMSS (9 year olds) / PISA

(14 year olds)

– Visual Arts and Music: New pilot tests designedin-house for the project

Standardised assessments we use for

pre- and post-testing

development and pedagogical intervention

Initial version of the rubric

CREATIVITY

(Coming up with ideas and solutions)

CRITICAL THINKING

(Questionning and evaluating ideas and solutions)

Progression

INQUIRE

Feel, empathise, observe, describe relevant experience and information

Explore, seek and generate ideas

Understand context/frame and boundaries of the problem

Review alternative theories and opinions and compare/find perspectives on the problem

Level 1 Level 2 Level 3 Level 4

IMAGINE

Make connections, integrate other disciplinary perspectives

Stretch and play with unusual/risky/radical ideas

Identify strengths and weaknesses of evidence, arguments, claims and beliefs

Challenge assumptions, check accuracy, analyse gaps in knowledge


DO / SHARE

Envision / Express / Produce / Prototype new product / solution / performance

Appreciate the novelty of solution and/or possible consequences

Appraise / Base / Justify opinion/products on logical, ethical or aesthetic criteria/reasoning

Acknowledge own bias (as perceived by others) and uncertainty/limits of endorsed opinion/solution


• Critical thinking is not (just):– Problem solving (= usually technical knowledge)

– Logical/rational thinking or understanding (weighing and justifying arguments)

• Most existing standardised tests of « critical thinking » assessunderstanding and logical thinking (ability to weigharguments within a given paradigm)

• It is about: – Challenging assumptions (the « core theory », not just auxiliary

hypotheses)

– Understanding the limitations of theories and conventions (live with ambiguity/uncertainty)

– Being able to consider other theories (perspective taking)

Some conceptual difficulties

• Creativity is not:

– Giftedness

– « New to the world » ideas

– Innovation (ideas/solutions with proven social/market value)

• Creativity is usually about:

– Originality + some level of usefulness/functionality

– Fluency + originality + elaboration

– Divergent-exploratory + convergent-integrative

• In our case

– Imagination, ideation

– Exploring unusual ideas

– Dare to be different

Some conceptual difficulties

• To develop new pedagogical activities

• To improve existing pedagogical activities

• To develop new rubrics (domain-specific, to assessstudennts, self-assessment, etc.)

• To assess student work

• To keep in mind the importance of thesecompetences (metacognition)

Uses of the rubric

Class-friendly rubric

CREATIVITY

(Coming up with ideas and solutions)

CRITICAL THINKING

(Questionning and evaluating ideas and solutions)

INQUIRING Play with unusual and radical ideas Challenge assumptions

IMAGINING• Generate ideas and make

connections• Find several perspectives on the problem

DOING Produce, perform or envision

something that is personally novel Propose own product/opinion justified

on logical, ethical or aesthetic criteria

REFLECTING

Assess the novelty of solution and of possible consequences

Acknowledge uncertainty/limits of chosen solution/position

Creative and critical thinking scaffolding

rubric (in progress)


Inquiring

Imagining

Reflecting

A pedagogical activity aligned with the creativity and critical thinking rubric should:

1. Create students’ need/interest to learn

2. Be challenging

3. Develop clear technical knowledge in one domain or more

4. Include the development of a product

What are the criteria for a good activity?

A pedagogical activity aligned with the creativity and critical thinking rubric should:

5. Have students co-design part of the product/solution or problem

6. Deal with problems that can be looked at from different

perspectives

7. Leave room for the unexpected

8. Include space and time for students to reflect and give/receive

feedback

What are the criteria for a good activity?

• Process and learning environment?– Montessori: Developmental Environmental Rating Scale (DERS)

– Developing a class observation tool to make sure we have a good teaching process?

• Student artefacts?– Should we notice in students’ outputs that they have more

creative and critical thinking potential?

– Could the process not be reflected in the actual output of students?

• Is there a possible contradiction between process and product?

– A creative educational process might not foster creative skills

– A directed educational process might foster creative skills

Some difficulties: how and what should

we assess?

preliminary findings(examples of outcomes of interest)

effects on teaching, learning

25

30

35

40

45

Control Intervention

Pre Post

-4.7

+5.3

Primary education:

I have to use my imagination

45

50

55

60

65


Pre Post

-3.4

+3.2

Primary education:

I have to make connections

between different school subjects

60

65

70

75

80


Pre Post

-4

+5.4

Primary education:

I have to look for several explanations

45

50

55

60

65


Pre Post

-0.8 +7.4

Primary education:

I do NOT only learn what I am interested in

Secondary education:

I have to use my imagination in this course

-1.2

+2.5

70

71

72

73

74

75

76

77

78

79


Pre Post

Based on HUN, NLD, THA


I have to solve problems that have more

than one possible solution

+0.8

+4.9

70

71

72

73

74

75

76

77

78

79


Pre Post



I have to produce or perform something on

my own

+2.2

+5.9

66

67

68

69

70

71

72

73

74

75

76

77


Pre Post



I have to explore different points of view on

a problem or topic in this course

-2.4

+3.4

68

69

70

71

72

73

74

75


Pre Post


Out of school effects

25

30

35

40

45


Pre Post

-11.0

-0.1

Primary education:

I try to explore new things

40

45

50

55

60


Pre Post

-6.5

+3.8

Primary education:

I am curious about many different things

standardised tests

50

55

60

65

70


Pre Post

+5.0+10.9

Primary education:

Performance in maths and science tests

60

62

64

66

68


Pre test VAM Post test VAM

+1.7

+3.47

Primary education:

Performance in visual art and music test


Performance in maths and science tests

-0.3

+2.1

40

45

50

55

Intervention Control

Pre score Post score

Based on HUN, NLD, USA-Montessori


Performance in visual arts test

-0.4

-2.8

55

57

59

61

63

65

Intervention Control

Pre score Post score

Based on NLD

teacher reports

50

55

60

65

70

75

80

85


Pre Post

+12.5

-0.32

Share of teachers who agree that creativity

can effectively be taught in schools

0

2

4

6

8

10

12

14

16

18

20


Pre Post

+4.75-3.33

Average number of times teacher ask students

to use everyday life examples during last month

0

2

4

6

8

10

12

14

16

18

20


Pre Post

+2.87-2.18

Average number of times teacher ask

students to explain the reasoning behind

their answers during last month

62.5 62.5

50

0

10

20

30

40

50

60

70

Preparation of lessons Interaction with students Pedagogical activities

Share of (intervention) teachers who report

they have changed their…

75 75

68.75

0

10

20

30

40

50

60

70

80

90

Students' motivation Students' enjoyment Promoting creativity skills

Share of (intervention) teachers who report

perceived positive effect on…

from formative to standardised assessment

• The innovative domain of PISA 2021 will be« creative thinking », building on the CERI projectand other sources

• To what extent do we need to change the usualassessment model – acknowledging that there willnot be more student time?

• International standardised assessments– Raise the profile of the objectives in policy making

– Have to remain connected to school teachingpractices

– Need to innovate in assesment format?

PISA 2021

concluding remarks

• Teachers

– Common understanding through a multiplicity of artefacts and tools (slow and long process)

– Need for changes in teaching (including assessment) practice and professional development

– Professional learning communities?

• Students

– More relevance of and engagement in their learning

– Acquisition of broader skill set

– Development of stronger social and behavioural skills

Concluding remarks

[email protected]

THANK YOU

www.oecd.org/edu/innovation

www.oecd.org/edu/internationalisation

www.oecd.org/edu/universityfutures

mailto:[email protected]

future learning skills in a global context: from … · « 21st century skills » ... domains or...

Documents