future-directed assessment: learning that lasts
DESCRIPTION
Presentation given at the MBA Directors' Forum, Brisbane, 3 May 2012TRANSCRIPT
![Page 1: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/1.jpg)
Future-directed assessment: Learning that lasts
Jeremy B Williamswww.jeremybwilliams.net
Chief Academic OfficerKnowledge Universe
Stamford Plaza Hotel, Brisbane3 May 2012
![Page 2: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/2.jpg)
2
Overview
1. The Five Minute University2. The case for authentic assessment3. Open-book, open-Web examinations4. A sample OBOW exam5. How to construct an OBOW exam6. Summary and conclusions
![Page 3: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/3.jpg)
3
1. The Five Minute University
![Page 4: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/4.jpg)
4
![Page 5: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/5.jpg)
5
2. The case for authentic assessment
![Page 6: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/6.jpg)
6
"Life is an open book exam."
• Learners need to be convinced of the authenticity of the task if they are to fully engage
Professor Alan Blinder
Princeton University
![Page 7: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/7.jpg)
7
• “... Engaging and worthy problems or questions of importance, in which students must use knowledge to fashion performances effectively and creatively.”
Authentic assessment defined
![Page 8: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/8.jpg)
8
Learning design for understanding
![Page 9: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/9.jpg)
9
• multiple-choice tests• fill-in-the-blanks• true-false• matching words• … Students are passive
learners surface learning
Authentic assessment is not:
![Page 10: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/10.jpg)
10
Does this resemble any real world setting?
![Page 11: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/11.jpg)
11
SPOT THE COMPUTER?
![Page 12: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/12.jpg)
12
Sound familiar?
![Page 13: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/13.jpg)
13
Assessment of learning
Content
Assessment
Learning outcomes
![Page 14: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/14.jpg)
14
Assessment for learning
Learning outcomes
Assessment
Content
![Page 15: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/15.jpg)
15
Closed book, invigilated exams are more likely to foster cramming/ data dumping than deep learning
![Page 16: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/16.jpg)
16
3. Open-book, open-Web (OBOW) exams
![Page 17: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/17.jpg)
17
In brief …
• A semi-structured ‘mini-case (or ‘caselette’)
• Harnesses the power of ICTs to emphasise currency and real world authenticity
• A summative assessment item …… invites the student to draw on all that they have learnt (determining what is relevant).
![Page 18: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/18.jpg)
18
Dull? Boring? Something to fear?
• Final assessment a celebration of learning!
• Boredom and stress not conducive to deep learning
• Important to catch the imagination and appeal to the creativity of the learner
• Student satisfaction is influenced by positive perceptions toward technology and an autonomous learning mode
(Drennan, Kennedy & Pisarski 2005)
![Page 19: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/19.jpg)
19
Key features
• Students play the role of decision-maker, auditor, consultant or advisor
• They are presented with a unstructured (open-ended) problem that requires resolution (usually in the form of a set of recommendations)
• No pre-exam night 'cramming'
![Page 20: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/20.jpg)
20
The template
• THE CONTEXT: the setting in which the problem/situation is identified and framed
• THE TASK: the project and issues to resolve
• THE GUIDE TO THE TASK: the setting of parameters and suggestions about methods/concepts/models/tools to employ.
![Page 21: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/21.jpg)
21
The ground rules• To minimise the scope for unethical behaviour …
1) Time period for the exam must be sufficiently tight
2) Make clear (as a stated objective of the subject) that demonstrable application of learning is the key to success
3) 'Text-book' impersonal responses will not attract high grades.
![Page 22: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/22.jpg)
22
4. A sample OBOW exam
![Page 23: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/23.jpg)
23
![Page 25: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/25.jpg)
25
1) Students cheat during invigilated exams
2) In the adult learner context, only a small percentage will attempt to cheat
3) These people will cheat whatever the exam instrument
Seldom observed points
![Page 26: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/26.jpg)
26
5. How to construct an OBOW exam
![Page 27: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/27.jpg)
27
Getting started
• Keep a look out for material all the time (not exam time!)
• e.g. Local newspaper, periodical websites, magazines, television news or current affairs programmes
![Page 28: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/28.jpg)
28
What to look for
• A story that learners can easily relate to in lay terms
• Objective: to get them to think deeply about an issue
• Student to act as ‘expert witness’ – an
effective mechanism for the validation of their learning in their own minds
![Page 29: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/29.jpg)
29
Creating a scenario
• Having settled on a theme, gather together various media that can bring the case to life
• The inclusion of hyperlinks, photographs and/or streaming media adds a human dimension authenticity
![Page 30: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/30.jpg)
30
Lead characters • No story is complete without lead
characters
• Using real people with names, and pictures and voices acts as a catalyst to student engagement
• Fictional characters must give the appearance of being real!
![Page 31: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/31.jpg)
31
Setting THE TASK
• Role play the bridge between a learner's education and their professional practice
• Placing the learner in the role of the key decision maker, the expert advisor, or the auditor serves to validate the student’s learning
• Revisit the stated learning outcomes … what skills should they have?
![Page 32: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/32.jpg)
32
Defining the parameters
• The definition of the assessment task might amount to no more than a paragraph
• Ideally it should invite a wide of variety of 'equally correct' answers
![Page 33: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/33.jpg)
33
Striking a balance
• Avoid 'spoon-feeding' but …
• … not so unstructured a student is either struck by 'writers block' or goes off in the wrong direction.
![Page 34: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/34.jpg)
34
Expectations• Before writing THE GUIDE TO THE TASK,
it is helpful to develop an outline of the kind of response one expects from the learner and, importantly, …
… how this aligns with the prescribed learning outcomes
• This process may also lead to THE TASK being refined
![Page 35: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/35.jpg)
35
6. Summary and conclusions
![Page 36: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/36.jpg)
36
OBOW exams …
• A form of assessment that fosters understanding of learning processes in terms of real-life performance … as opposed to a display of inert knowledge
• Test problem-solving skills not memory
• Equips learners with 21st Century skills
![Page 37: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/37.jpg)
37
What OBOW exams deliver…
• An assessment instrument that is more relevant to goals of the curriculum, greater authenticity, where real-world problems take centre-stage
• Allow ICTs to be harnessed to encourage interaction
• Student engagement with the assessment task induces deeper learning
• Low cost solution for exam delivery in open and distance learning
![Page 38: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/38.jpg)
38
• Studies show stimulation with audio will increase retention rate by 20%. If stimulated with audiovisual, memory retention climbs to 30%. If presented with interactive multimedia involvement, the retention rate can be as high as 60%.
-- Tay Vaughan from Multimedia: Making it Work, 5th edition, 2001
![Page 39: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/39.jpg)
39
References• Williams, Jeremy B. (2009)
The efficacy of the final examination: a comparative study of closed-book, invigilated exams and open-book, open-web exams (with Amy Wong), British Journal of Educational Technology, 40 (2), 227-236).
• Williams, Jeremy B. (2007) E-xams: harnessing the power of ICTs to enhance authenticity, (with Wing Lam and Alton Chua), Educational Technology and Society, 10 (3), 209-221.
• Williams, Jeremy B. (2007) Using digital storytelling as an assessment instrument: Preliminary findings at an online university, (with Kanishka Bedi), Proceedings of the 11th CAA Conference, pp.433-447, Loughborough, England, 10-11 July.
• Williams, Jeremy B. (2006) The place of the closed book, invigilated final examination in a knowledge economy, Educational Media International, 43(2), 107-119.
![Page 40: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/40.jpg)
CreditsAlan Blinder, http://halleinstitute.emory.edu/images/research/blinder_largeGrant Wiggins, http://myworldpearson.com/images/wiggins.jpgMCQs, http://www1.istockphoto.com/file_thumbview_approve/2856522/2/istockphoto_2856522_multiple_choice_exam.jpgExam halls, http://i.telegraph.co.uk/multimedia/archive/01529/exam-hall_1529387c.jpg~ http://www.admin.ox.ac.uk/schools/graphics/ewert2.JPG~ http://picasaweb.google.com/lh/photo/SQORvbJYKhErQ67sYQgdVg~ http://qixsaliva.blogspot.com/2007/04/final-destination-2.html~ http://www.fotosearch.com/thumb/DGV/DGV078/200239868-001.jpgBruce Wellman http://pairadimes.davidtruss.com/question-everything/Students will cheat http://i36.photobucket.com/albums/e47/priyankashis/cheating.jpgCheating http://i36.photobucket.com/albums/e47/priyankashis/cheating.jpg
![Page 41: Future-directed assessment: Learning that lasts](https://reader036.vdocument.in/reader036/viewer/2022062418/555c9962d8b42a85758b5026/html5/thumbnails/41.jpg)
41
authenticlearning.wordpress.com
@jeremybwilliams
http://sg.linkedin.com/in/jembwilliams
http://www.slideshare.net/jembwilliams