Roles and types of examination

ITU, October 8, 2010

Hanne Leth Andersen, RUC

“Assessment methods [….] probably have a greater influence on how and what students learn than any other single factor.” (Boud 1988)

“Students can avoid bad teaching: they can’t avoid bad assessment.” (Boud 1994)

FrameworkExam systems and national policies

• Institution based or externally controlled exams• Authority: testing agency or ministry• Assurance of independency and criterion based evaluation• Assessment and censorship

• Oral and written exams: finance and reliability

Exam typology: Parameters

1. Medium: oral – written – combination 2. Control: set questions – free – combination 3. Approach: problem based – syllabus based4. Materials allowed: none, certain, notes, all5. Preparation: without – with (+ time)6. Examination time7. Number of students: individual – group8. Evaluator: teacher – co-examiner – student9. Focus: process/product/presentation orientation

Exam types

• Oral, written and practice related assignments (one or several)

• Participation in teaching, courses, seminars, ...• Combinations of 1-2• Practice

University act, Examination order, §3,2

Criteria for choosing exam forms

• Relevance– The goal is to judge if and to what extend student

qualifications are in accordance with objectives, competences and academic requirements established in the Ministerial Order on Education, the study program etc. (Ministerial order on Examinations, §2)

• Validity– Alignment between formulated objectives, measurable

operational demands at exams and assessment criteria• Reliability

– Precise and consistent (fair) assessment (oral vs. written)• Economy

Ressources

• Mono-dimensional exam forms: basic skills and knowledge• Poly-dimensional exam forms: overall (often multi

disciplinary) objectives (analysis, synthesis)• Combined exam forms

Uddannelsesstyrelsen 2004

Competences as learning objectives

• Critical approach• Independence and autonomy • Collaboration• Structuring and planning• Problem solving and project work • Oral and written presentation• Creativity• Peer and self evaluation

Alignment

• Unaligned Course • Aligned Course

Teacher’sintention

Student’sactivity

Examassessment

Teacher’sintention

Student’sactivity

Examassessment

e.g.- explain- relate- prove- apply

e.g.- memorize- describe

e.g.- memorize

From Claus Brabrand, ITU

Relevant, modern, authentic exam forms

• Portfolio• Synopsis• Case• Project • Group preparation • Practice based exams• Peer Assessment

Portfolio: collection, reflection, selection

• Systematic collection of works -> expression of development and learning

• Variation in materials and assignments• Electronic portfolio• Process orientation• Reflexion, consciousness about learning• Coverage

Portfolio and feedback

• ”As an indicator of achievement, portfolios were designed specifically as an opportunity to provide feedback, assuming an active, even steering role of the learner in the assessment task”

• Smith & Tillema, 2003, s. 626

Oral exam with synopsis

• Research question or problem formulation• Problems, discussion of materials, theories and methods• Conclusions• Literature• Perspectives

• NB: Reliability and resources

Case exams: authenticity

• Point of departure: a “real” situation• Available data and materials: “real”• Preparation and evaluation• Objective: identify, analyse and solve problems• Use of theories while working with the case • Case based exam forms useful to test problem solving

competences in specific contexts

Case: Complexity dimensionsAnalytic dimension• Questions• Problem definition and formulation• Type of analysis needed • One or several solutions

Complexity• Which theories and models can be used – and how difficult

are they?• Do they belong to the same discipline?• To which degree must several disciplines be integrated?

Presentation level• Is relevant information accessible?• Which important information is not included?• Is information structured?

Project exams

• Problem based learning• Within a restricted area be able to formulate a problem that

makes sense• Beyond declarative knowledge• Focus on functional knowledge• Specific knowledge directed towards practice• Autonomy • Explicitly include development of personal and social

competences• Project management• -> demanding work form

Examination skills

• Dialogue at oral exams• Collaboration and question typologies• Good questions• Level adaptation• Written exam questions• Censor’s role

Training

• Collegial supervision as a tool • Conversation analysis• Question typologies• Video + reflection• Censor’s observations• Exam testing with colleagues or students

References• Andersen, H.L. & J. Tofteskov, 2008. Eksamen og

eksamensformer. Betydning og bedømmelse, Samfundslitteratur.• Biggs, J. 1987. Student Approaches to Learning and Studying,

Hawthorn, Victoria: Australian Council for Educational Research.• Bloom, B.S. (ed.) 1956. Taxonomy of educational objectives: The

classification of educational goals: Handbook I, cognitive domain, David McKay, New York.

• Brown, S. Rust, C. & Gibbs, G. 1994. Strategies for Diversifying Assessment in Higher Education. Oxford Centre for Staff Development.

• Dysthe, O. & K.S. Engelsen 2007. “Variations in higher education portfolio assessment. A nationwide survey in Norway across institutions and disciplines”. Norgesuniversitetets konferanse om e-portfolio.

• Uddannelsesstyrelsen 2004, Modernisering af prøver, eksamener og karakterer.