ITS4SEA

Technology-enhanced examinations at the University of BradfordJohn Dermo, Centre for Educational Development,University of BradfordUniversity of Edinburgh 23rd April 2012

#24/04/20121Technology enhanced exams at UoBA strategic approach to technology enhanced assessmentDrivers: including the student perspectiveDeveloping a facility for e-assessmentInnovative e-assessment at UoB

#

e-Learning Pathfinder ProgrammeEmbedding Support Processes for e-AssessmentPathfinder Project 2007-8http://www.bradford.ac.uk/educational-development/educational-research/pathfinder-caa/#24/04/20123

JISC Institutional Exemplar ProjectIntegrating thin client systems and smart card technology for flexible, accessible and secure E-assessment.

ITS4SEA Project 2007-9

http://www.jisc.ac.uk/whatwedo/programmes/institutionalinnovation/its4sea.aspx

Institutional Innovation Programme#Drivers: reasons for focus on online assessment University e-Strategy (2004-9)LTA strategy and Academic Framework (2009-14)National Student SurveyIncreased use of e-learningStudent expectations

#In 20 years HE has seen radical and unprecedented change (Education Act, 1992; Dearing Report, 1997; Roberts Report, 2003; Leitch Report, 2006; Browne, 2010; CSR, 2010)Learners have different expectations and assumptions about their HE experiencesThe student body has become dramatically more heterogeneousImpact of student fees#NATIONAL UNION OF STUDENTS PRINCIPLES OF EFFECTIVE ASSESSMENT (2009)Should be for learning, not simply of learningShould be reliable, valid, fair and consistentShould consist of effective and constructive feedbackShould be innovative and have the capacity to inspire and motivateShould measure understanding and application, rather than technique and memoryShould be conducted throughout the course, rather than being positioned as a final eventShould develop key skills such as peer and reflective assessmentShould be central to staff development and teaching strategies, and frequently reviewedShould be of a manageable amount for both tutors and studentsShould encourage dialogue between students and their tutors and students and their peers#

Please, Sir, may I have some more exams?Becka Colley, Dean of Students, University of Bradford. Keynote speech at eAssessment Scotland, 2011http://www.slideshare.net/rcurrant/please-sir-may-i-have-some-more-examsWith apologies to Charles Dickens and Ronald Searle#24/04/20128There was a strong preference to retain two exam sessions. This was felt to be more manageable in terms of revision. Students were not in favour of one exam session at the end of the academic year. A staged approach with summative assessment was viewed as the most appropriate. Also, January exams give an opportunity for students to trial their exam revision and writing skills. [UBU feedback on review of the academic year, June 2011]

Student Union Feedback#Dedicated supportMove towards integrated processes and systemsWorkflow model and clear roles and communication channelsApproved policiesTraining materials and sessions, briefingsDevelop new flexible space for e-assessment

Developments at UoB

#100 seat e-assessment cluster

#24/04/201211

#24/04/201212Smartcards: link each exam to a different card

VMware and Sun Ray servers

#Benefits of e-assessment suite for online assessmentsCan re-image software available very quickly between examsEasy to control software available, flexible using Appsense Can link assessment to card to allow multiple simultaneous exams in single locationEasier to invigilate more secureRoom easier to findMore pleasant environmentEasier to manage extra time students

#24/04/201214First used January 2009, since when the use for summative e-assessment has gradually increased:

Usage: 2500 different students per year, from more than 30 modules covering range of subject areas across disciplines

Ive already told the DVC that we will need another cluster like this within 2 yearsAssociate Dean (Teaching and Learning) Life SciencesThis is just what I always wanted but we couldnt do on the PCs!Lecturer in School of Management#24/04/201215Growing interest in e-assessmentFurther demand on facilities, especially for summative assessmentMore efficient use of space (smartcards, flexible scheduling)Overflow room to increase capacity to 130Single rooms available when necessary

#24/04/201216Some Student Feedback#Varied use of e-assessment suite

exam periods: block booked by exams office for summative e-assessment term time managed by room bookings for one off e-assessments regular teaching sessions otherwise: open 24/7 as self access cluster for students

#24/7 Cluster UseRoom in regular usevibrant environmentPleasant learning spaceEncourages group workDemand during exam periodBut limitations of thin-clients?

#Challenges

How to manage the resource 365 days a year?Who owns the facility?How to get the most out of the investment?How to manage larger groups?Booking the room?Server capacity to allow 100+ logins? Expanding the use of the room to include essay exams (cf interest in e-submission and e-marking)

#Integrating Institutional Processes

#24/04/201221But dont forget the human processes which go with the technical systems

#Innovation requires commitment and acceptance of responsibility across the institution

instructors course teams invigilators students department adminexams office disabilities office senior managementIT servicessupport teamsroom bookingsline managersestates#Lessons Learned at Bradford?Challenges are not just technology-basedCommunication is keyYour systems are only as good as the data in themAutomated systems require humans to play their part#We all know what an exam looks like, dont we??

From thisto this.#Examples of authenticityDepartment of Biomedical Sciences: Combination of web-based genetic research tool (BLAST) with MCQs to deliver authentic, real-life tasks which require application of knowledge and understanding in a real-life professional contextCan be assessed automatically = great for large numbers of students #School of Medical Sciences: Use of formative e-assessment for learning (in F42 and via VLE) to deliver automatic generic feedback to students. Work has been carried out on delivering question level feedback and topic based feedback students would appear to best appreciate a combination of these. Examples of authenticity#School of Optometry: Use F42 for Active problem-based learning Formative group based case studiesInformation is automatically and gradually released during processStudents make joint decisions, then receive email feedback. Group focus initially then individual summative processExamples of authenticity#Hybrid assessments

Some questions automatically marked, others done by a human

More efficient use of human resources to allow time to be spent on students and feedback processes.Other Developments#School of Health Studies, School of Management and School of Lifelong Education and Development: Using e-peer assessmentStudents grading performance and/or give feedback collaborative group tasks. This assessment can take place either all together in the e-assessment suite or remotely via the VLE.

Other Developments#Activity-based Student-centred (AbSc) teaching and learning: an Engineering perspective at Bradford 8 e-Assessments in 12 weeks in F42 (including 6 bi-weekly assessments, one e-assessment on lab work and one mid-term e-assessment) Supported by online quizzes self assessment with feedback done in own time in VLE Final summative e-assessment in F42

Other Developments#F42

#Thank You

Any questions?

j.dermo@bradford.ac.uk

#24/04/201233