The relation between assessment

and learning

Prof. dr. M. Segers. University Leiden (NL)

Wibran and the music classWibran and the music class

‘‘Assessment steers learning’Assessment steers learning’

Negative effects of traditional testsNegative effects of traditional testsTheorized benefits of new modes of Theorized benefits of new modes of

assessmentassessment

Traditional tests:Traditional tests: referred to as measuring low referred to as measuring low levels of comprehension; negative effects on levels of comprehension; negative effects on various motivational aspects (Harlen & Crick, various motivational aspects (Harlen & Crick, 2001 ) and stimulating the use of surface 2001 ) and stimulating the use of surface learning approachlearning approach

new modes of assessment:new modes of assessment: measuring measuring knowledge construction & application, problem-knowledge construction & application, problem-solving, reflection & critical analysis, going solving, reflection & critical analysis, going beyond mere facts, relating,…. Supporting the beyond mere facts, relating,…. Supporting the use of a deep approach to learninguse of a deep approach to learning

With the implementation of new modes of With the implementation of new modes of assessment, students are stimulated to focus on assessment, students are stimulated to focus on

these goals. It is expected that, in comparison these goals. It is expected that, in comparison with traditional tests, this will lead to students’ with traditional tests, this will lead to students’ adoption of a deep approach to learning and adoption of a deep approach to learning and

considerable learning gainsconsiderable learning gains

Pre-assessment effectPre-assessment effect

Although the statement ’assessment steers Although the statement ’assessment steers learning’ has stimulated many schools to learning’ has stimulated many schools to change their assessment practices, the change their assessment practices, the empirical evidence of the effect of these empirical evidence of the effect of these changes on student learning is scarce.changes on student learning is scarce.

3 empirical studies3 empirical studies Segers, Nijhuis & Gijselaers: effect of redesign Segers, Nijhuis & Gijselaers: effect of redesign

of learning and assessment environment on of learning and assessment environment on students’ learning strategies (International students’ learning strategies (International Business)Business)

Segers, Gijbels & Thurlings: The effect of Segers, Gijbels & Thurlings: The effect of perceptions of the assessment practice on perceptions of the assessment practice on student learning approaches. The case of student learning approaches. The case of portfolio assessment (Applied Science)portfolio assessment (Applied Science)

Segers, Martens, Nijhuis, Cats & Verwijmen: the Segers, Martens, Nijhuis, Cats & Verwijmen: the role of assessment perceptions and prior role of assessment perceptions and prior experiences with assessment on student experiences with assessment on student learning approaches (Teacher Training program)learning approaches (Teacher Training program)

Research modelResearch model

Perceptions

Of the assessment demands

Of the assessment practice

Prior experiences

Intentional learning approaches

Personal beliefs on assessment

Learning approaches

Segers et al: effect of redesign of learning and Segers et al: effect of redesign of learning and assessment environment on perceptions of the assessment environment on perceptions of the assessment demands and learning strategiesassessment demands and learning strategies

Redesign from assignment-based format (ABL) Redesign from assignment-based format (ABL) to a PBL format: to a PBL format:

(1) from clear-cut study assignments to authentic (1) from clear-cut study assignments to authentic business problems, business problems,

(2) from passive participation to active (2) from passive participation to active discussion; discussion;

(3) from one handbook to multiple literature (3) from one handbook to multiple literature resourcesresources

(4) assessment: from knowledge reproduction (4) assessment: from knowledge reproduction (ABL) to use of knowledge as a tool to solve (ABL) to use of knowledge as a tool to solve novel problems (PBL)novel problems (PBL)

Case Mexx/Benneton(This case of approx. 35 pages is new the students)

The case study presents the history and recent developments in the fashion company Mexx. Main trends within the European clothing industry are described. Mexx Fashion and Benneton are pictured by their organizational structure, the product profile and marketplace, the business system, the corporate culture and some actual facts and figures.

Question1true/?/false Mexx’s corporate culture and philosophy is consistent with the systems viewpoint on management.(False, it is consistent with behavioural viewpoint)

Question 2Benneton’s and Mexx’s corporate strategies are quite different. More specifically, there are two main differences.•Identify these two main differences in corporate strategies. Illustrate your answer with examples mentioned in the case.•What are the advantages of Benneton’s corporate strategy compared to Mexx’s approach?

Research question: Effect of redesign Research question: Effect of redesign on students’ learning strategies? on students’ learning strategies? Relation with intentional learning Relation with intentional learning

strategies and perceptions of strategies and perceptions of assessment demands?assessment demands?

Actual Learning strategies: Actual Learning strategies: R-SPQ-2FR-SPQ-2F (Biggs) (Biggs)

Surface strategy: I learned some things by rote, going over them until I Surface strategy: I learned some things by rote, going over them until I knew them by heartknew them by heart

Deep strategy: I related material, as I was reading it, to what I already Deep strategy: I related material, as I was reading it, to what I already knew on that topic knew on that topic 

Intentional Learning strategies:Intentional Learning strategies: R-SPQ-2FR-SPQ-2F (Biggs) (Biggs)

Surface strategy: I intend to learn things by rote, going over them until I Surface strategy: I intend to learn things by rote, going over them until I know them by heartknow them by heart

Deep strategy: I intend to relate material, as I am reading it, to what I Deep strategy: I intend to relate material, as I am reading it, to what I already know on that topic already know on that topic 

Perceptions of Assessment Demands (Scouller)Perceptions of Assessment Demands (Scouller)

Assessment of surface learning skills: I expect the test to assess my Assessment of surface learning skills: I expect the test to assess my ability to reproduce key terms and definitions ability to reproduce key terms and definitions 

Assessment of deep learning skills I expect the test to assess my Assessment of deep learning skills I expect the test to assess my ability to integrate from a variety of sources ability to integrate from a variety of sources 

ABL PBL P

Actual deep learning strategies

23.50 (3.28) 22.22 (3.48) 0.000

Actual surface learning strategies

16.54 (3.11) 17.98 (2.93) 0.000

RESULTS

Result 1:

less deep learning in OAT-condition

ABL condition Actual deep LS

Deep perception assessment demands

Intention to employ deep learning strategies

.433** .290**

PBL condition

Intention to employ deep learning strategies

.428** .334**

Result 2:

Influence of prior experiences

Segers et al: the relation between learning Segers et al: the relation between learning approaches, perceptions of the assessment approaches, perceptions of the assessment

demands and personal beliefs on assessment demands and personal beliefs on assessment demandsdemands

Context: 4 year teacher training program; Context: 4 year teacher training program; year 1& 2 knowledge reproduction tests; year 1& 2 knowledge reproduction tests; year 3 OverAll Tests; year 4 portfolio year 3 OverAll Tests; year 4 portfolio assessment assessment

Research question: what is the relation Research question: what is the relation between student between student learning approacheslearning approaches and and their their perceptions of assessment demandsperceptions of assessment demands and and personal beliefs on assessment personal beliefs on assessment demandsdemands??

Personal beliefs of assessment demandsPersonal beliefs of assessment demands

Belief of surface assessment: In general I believe a test Belief of surface assessment: In general I believe a test assesses my ability to reproduce key terms and assesses my ability to reproduce key terms and definitionsdefinitions

Belief of deep assessment: Belief of deep assessment: In general I believe a test In general I believe a test assesses my ability to integrate from a variety of assesses my ability to integrate from a variety of sourcessources

Surface approach

Deepapproach

Surface percep.

Deep percep

Beliefs surf.

Beliefs deep

Surface approach

-,510** -,126 -,321* -,083 -,159

Deep approach

,188 ,215 ,309* -,003

RESULTS

Result 1:

Some association between approaches and perceptions, beliefs

PERCEPTIONS

SURFACEPERCEPTIONS

DEEP

BELIEFS SURFACE,544(**) -,004

BELIEFS DEEP-,051 ,604(**)

Result 2: role prior experiences

A strong positive association between perceptions of assessment demands and personal beliefs

Segers et al: The effect of perceptions of the Segers et al: The effect of perceptions of the assessment practice on student learning assessment practice on student learning

approaches. The case of portfolio assessmentapproaches. The case of portfolio assessment

Context: 4 year program Applied Science; Context: 4 year program Applied Science; competency-based; portfolio assessment competency-based; portfolio assessment

Digital portfolio: PAP, PDP, study dossier (work Digital portfolio: PAP, PDP, study dossier (work done and work in progress) and private dossier ; done and work in progress) and private dossier ; basis for the assessment dossier for end-of-year basis for the assessment dossier for end-of-year assessmentassessment

Research question: is there a relation between Research question: is there a relation between students’ students’ perceptions of the portfolio perceptions of the portfolio assessmentassessment and their and their learning approacheslearning approaches??

Scale Item examples

1. Amount and distribution of student effort

I only study things that are going to be covered in the assignments; On this course it is possible to do quite well without studying too much

2. The portfolio assessment and learning

Making the portfolio brought things together for me; The instructions are very clear about what you are expected to do for the portfolio assessment; The portfolio assessment is not very challenging

3. Quantity and timing of feedback

On this course I get plenty of feedback on how I am doing; The feedback comes back very quickly

4. Quality of feedback The feedback helps me to do things better; I can seldom see from the feedback what I need to do to improve

5. What you do with the feedback

I use the feedback to go back over what I have done in the assignment; I tend to only read the marks

Perceptions of portfolio assessment (Gibbs Assessment Perceptions of portfolio assessment (Gibbs Assessment Experiences Questionnaire, 2994)Experiences Questionnaire, 2994)

2 3 4 5 6 7

1. Amount and distribution of effort

-,118 -,220* -,128 -,080 0.097 0.097

2.Portfolio Assessment and learning

,354** ,402** ,383** 0.320** -0.267**

3. Quantity and timing of feedback

,337** ,189* 0.102 -0.236*

4. Quality of feedback

,317** 0.043 -0.269**

5. Student’s use of feedback

0.371** -0.395**

6. Deep learning approach

-0.424**

7. Surface learning approach

RESULTS

Results:Results:

Students’ perceptions of the assessment Students’ perceptions of the assessment practice are related to their approach to practice are related to their approach to

learning,learning,

more specifically,more specifically,

how they perceive the feedbackhow they perceive the feedback

andand

how hey perceive the relation between how hey perceive the relation between assessment and learningassessment and learning

Overall, what do these studies Overall, what do these studies indicate?indicate?

Simply changing the mode of assessment does Simply changing the mode of assessment does not always result in changes in learning not always result in changes in learning

approachesapproaches

Personal history of students plays a role: Personal history of students plays a role: (1) intentional learning strategies(1) intentional learning strategies(2) personal beliefs on assessment(2) personal beliefs on assessment

Role of students’ perception of assessment Role of students’ perception of assessment practicespractices

Implications for future researchImplications for future research

A challenge for future research is to gain insight A challenge for future research is to gain insight into the choices a student makes when trying to into the choices a student makes when trying to deal with the various individual and contextual deal with the various individual and contextual elements and the demands of the learning and elements and the demands of the learning and assessment environment: the interplay betweenassessment environment: the interplay between

prior experiences, prior experiences, perceptions and appraisal of assessmentperceptions and appraisal of assessment learning approacheslearning approaches

Implications for educational Implications for educational practicepractice

Give student time and opportunities to practice Give student time and opportunities to practice and to have positive experiencesand to have positive experiences

Talk to students and observe them in order to Talk to students and observe them in order to gain insight in how student deal with the gain insight in how student deal with the assessment practice in order to fine-tune assessment practice in order to fine-tune learning and assessment practiceslearning and assessment practices

Organize enough and on-time feedback and give Organize enough and on-time feedback and give students opportunities to work with the students opportunities to work with the suggestions they have receivedsuggestions they have received

Make the assessment a learning activity by Make the assessment a learning activity by giving them challenging assessment tasks and giving them challenging assessment tasks and making the assessment process transparentmaking the assessment process transparent

Or…Or…

Overcoming assessment challenges!