running an exam from beginning to end - university of bristol
TRANSCRIPT
![Page 1: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/1.jpg)
Running an Exam From Beginning to End
Andrew Blythe
Autumn 2011
![Page 2: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/2.jpg)
BLUEPRINTING
What are we assessing?
![Page 3: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/3.jpg)
“The outcomes in Tomorrows Doctors (2009) provide a structure against which everything else can be planned & evaluated. Some outcomes cover more ground than others so will require more extensive testing.”
Paragraph 29, Assessment in Undergraduate Medical Education – Advice
Supplementary to Tomorrow’s Doctors (2009)
![Page 4: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/4.jpg)
Hierarchy of Learning Outcomes
Unit Intended Learning Outcome Approved by MEC & Programme Approval
MB ChB Programme Specification University of Bristol Under revision
GMC 16 high level outcomes 32 practical procedures
![Page 5: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/5.jpg)
Learning Outcomes for Preparing for Professional Practice (Year 5)
Examples Tomorrows
Doctors Brief description topic/learning objective
Method of Assessment
14a Practitioner
Synthesises history, examination, mental state & patient factors
Formative & summative long cases Portfolios Case-based discussion within tutorial
17a 17b Practitioner
Takes a drug history including OTC medications & complementary therapies & plan appropriate therapy for common indications, including pain & distress
Examples of appropriate drug histories in clerking portfolio with copies of original drug charts Student script countersigned Oncology/palliative care prescribing tutorial signed off
![Page 6: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/6.jpg)
Community Orientated Medical Practice 2 (COMP2)
Blueprint for Primary Care
![Page 7: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/7.jpg)
ROLE OF EXTERNAL EXAMINER
![Page 8: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/8.jpg)
External Examiner from Report Received
Human Basis of Medicine Oxford Yes
Molecular & Cellular Basis of Med
Cardiff, Kings College + 2 others
Yes (2 out of 4)
Systems of the Body 1 & 2 Kings College Yes
Junior Medicine & Surgery
Cardiff + 1 other Yes (1 out of 2)
Psychiatry & Ethics Southampton & Hull-York Yes (both)
MDEMO 1 No
COMP1 Highlands + 2 Yes (2 out of 3)
Applied Clinical Sciences 2 No
Reproductive Health Care Swansea & Penninsula Yes (both)
COMP2 Kings College & Sheffield Yes (both)
Senior Surgery Oxford No
Oncology & Palliative Care No one in place
Senior Medicine Birmingham Yes
SSCs (years 3 & 5) 1 No (2nd year running)
![Page 9: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/9.jpg)
University Guidelines
“The role of the external examiner is not confined to consideration of examination results and attendance at examination boards”.
“External examiners may, and are encouraged, to comment and advise on the content, balance and structure of the programmes and units, the development and review of programmes and/or units and on assessment processes.”
![Page 10: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/10.jpg)
The External Examiner’s Annual Cycle
Review content & assessment of unit. Help set priorities for coming year
Comment on proposed
examination questions
Contribute to standard setting
exercise
Attend examination & scrutinize
student performance
Submit report
Report viewed by Unit Assessment Group APR Response sent to External Examiner
![Page 11: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/11.jpg)
Planning Changes
Reviewing Questions
Standard Setting
Attends Exam
Reviews marks of failing students
HBoM
MCBoM
Systems 1
Systems 2
Junior Med & Surg OSCE
Psych & Ethics Presentations
MDEMO
COMP1 Prize viva
ACS
RHCN SSC Pres’n
COMP2 OSCEs & SSC presentation
Senior Surgery
Senior Medicine OSCE & resit long cases
![Page 12: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/12.jpg)
Problems
Long delays in
• getting some reports
• reviewing & responding to some reports
Not involving external examiners enough in
• reviewing some units
• looking at performance of students
![Page 13: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/13.jpg)
Examination Format: written
• Replace essays with more short answers
• Replace true/false with best-of-5
• Standardise Extended Matching Questions
• Base majority of questions on clinical topics
![Page 14: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/14.jpg)
Examination Format: clinical
• More OSCE stations where student has to examine the patient/actor
• Standardise assessments
• Set standard for portfolios
![Page 15: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/15.jpg)
Communication with External Examiner
• Discuss course details with him/her
• Discuss student performance with him/her
• Inform him/her of student progress
![Page 16: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/16.jpg)
Other comments
• Concern about method of scaling marks
• Some unit leads need more support
![Page 17: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/17.jpg)
TRAINING EXAMINERS
![Page 18: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/18.jpg)
Doves & Hawks
![Page 19: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/19.jpg)
![Page 20: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/20.jpg)
STANDARD SETTING
![Page 21: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/21.jpg)
Written Clinical
Year 1 HBoM: Team of trained examiners mark essays. Check-lists created
MCBoM: Trial standard setting group spanning all 5 academic departments in Faculty MVS
Year 2 Ebel method used for written papers Mock standard setting exercise for spot anatomy
Year 3 Ebel for Junior Medicine & Surgery Borderline Group Method for OSCE in Junior Med & Surgery
Year 4 Full implementation of Ebel in all 4 units Borderline Group Method for OSCEs.
Year 5 Full implementation of Angoff in EMQs for Finals
Angoff for DOSCE stations Borderline Regression Method for OSCE stations
SSCs
![Page 22: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/22.jpg)
Competent
Pass
Incompetent
Pass
Competent
Fail
Incompetent
Fail
![Page 23: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/23.jpg)
4 components to COMP2 exam
• OSCE 10 stations (each out of 10)
• MCQ (negatively marked)
• Student selected component (essay & group presentation)
• Satisfactory professional behaviour
![Page 24: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/24.jpg)
4 components
• OSCE 10 stations (each out of 10)
• MCQ (negatively marked)
• Student selected component (essay & group presentation)
• Satisfactory professional behaviour
50%
50%
45%
Pass/fail
![Page 25: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/25.jpg)
Marks in an OSCE
10 20 30 40 50 60 70 80 90 100
No. of students
Pass mark
Total mark in OSCE (%)
![Page 26: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/26.jpg)
The marks of a poorly performing student
Station
A B C D E F G H I J
4 4 6 5 5 4 4.5 7 6 5 50.5
![Page 27: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/27.jpg)
New rule:
“Students should get more than 50% overall & should not fail more than 2 stations”
Station
A B C D E F G H I J
4 4 6 5 5 4 4.5 7 6 5 50.5
FAIL
![Page 28: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/28.jpg)
EXAMINER FORM STICK STUDENT ID LABEL HERE
Check that you have correct
student’s name
and number on the marking sheet.
This station is testing the ability of the student to
Manage a patient with mechanical low back pain Elicit the patient’s expectation of a consultation
COMPETENCE TASK SCORE
Competence task (Maximum mark) Mark ( ½ marks allowed) Comment (only if required)
Student takes a good history including the effect of the
symptoms on the patient’s work (2)
Student checks for red flag symptoms: bladder, bowels & saddle
numbness (1 ½)
Student explores patients concerns & finds out that patient was
expecting an X-ray; student explains why this is not a good
idea (2 )
Student explains natural history of low back pain (1)
Student negotiates management plan with patient. This should
using a stronger analgesic drug, encouraging patient to mobilise
and possibility of self certification/ Med3 (2)
Student gives advice on safe lifting (1 ½ )
TOTAL (out of 10)
OVERALL IMPRESSION How do YOU feel the student has performed? Please circle the description that best describes the performance of this student:
FAIL BORDERLINE
PASS
CLEAR PASS
![Page 29: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/29.jpg)
EXAMINER FORM STICK STUDENT ID LABEL HERE
Check that you have correct
student’s name
and number on the marking sheet.
This station is testing the ability of the student to
Manage a patient with mechanical low back pain Elicit the patient’s expectation of a consultation
COMPETENCE TASK SCORE
Competence task (Maximum mark) Mark ( ½ marks allowed) Comment (only if required)
Student takes a good history including the effect of the
symptoms on the patient’s work (2) 1
Student checks for red flag symptoms: bladder, bowels & saddle
numbness (1 ½) 1
Student explores patients concerns & finds out that patient was
expecting an X-ray; student explains why this is not a good
idea (2 )
½
Student explains natural history of low back pain (1) ½
Student negotiates management plan with patient. This should
using a stronger analgesic drug, encouraging patient to mobilise
and possibility of self certification/ Med3 (2)
1½
Student gives advice on safe lifting (1 ½ ) ½
TOTAL (out of 10) 5
OVERALL IMPRESSION How do YOU feel the student has performed? Please circle the description that best describes the performance of this student:
FAIL BORDERLINE
PASS
CLEAR PASS
![Page 30: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/30.jpg)
Borderline Group Method C
om
pet
en
ce
task
sco
re
10
6.1
0
Clear Borderline Clear Very Outstanding
Fail Pass Good
![Page 31: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/31.jpg)
EXAMINER FORM STICK STUDENT ID LABEL HERE
Check that you have correct
student’s name
and number on the marking sheet.
This station is testing the ability of the student to
Manage a patient with mechanical low back pain Elicit the patient’s expectation of a consultation
COMPETENCE TASK SCORE
Competence task (Maximum mark) Mark ( ½ marks allowed) Comment (only if required)
Student takes a good history including the effect of the
symptoms on the patient’s work (2)
Student checks for red flag symptoms: bladder, bowels & saddle
numbness (1 ½)
Student explores patients concerns & finds out that patient was
expecting an X-ray; student explains why this is not a good
idea (2 )
Student explains natural history of low back pain (1)
Student negotiates management plan with patient. This should
using a stronger analgesic drug, encouraging patient to mobilise
and possibility of self certification/ Med3 (2)
Student gives advice on safe lifting (1 ½ )
TOTAL (out of 10)
OVERALL IMPRESSION How do YOU feel the student has performed? Please circle the description that best describes the performance of this student:
FAIL BORDERLINE
PASS
CLEAR PASS VERY GOOD OUTSTANDING
![Page 32: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/32.jpg)
EXAMINER FORM STICK STUDENT ID LABEL HERE
Check that you have correct
student’s name
and number on the marking sheet.
This station is testing the ability of the student to
Manage a patient with mechanical low back pain Elicit the patient’s expectation of a consultation
COMPETENCE TASK SCORE
Competence task (Maximum mark) Mark ( ½ marks allowed) Comment (only if required)
Student takes a good history including the effect of the
symptoms on the patient’s work (2) 1
Student checks for red flag symptoms: bladder, bowels & saddle
numbness (1 ½) 1
Student explores patients concerns & finds out that patient was
expecting an X-ray; student explains why this is not a good
idea (2 )
½
Student explains natural history of low back pain (1) ½
Student negotiates management plan with patient. This should
using a stronger analgesic drug, encouraging patient to mobilise
and possibility of self certification/ Med3 (2)
1½
Student gives advice on safe lifting (1 ½ ) ½
TOTAL (out of 10) 5
OVERALL IMPRESSION How do YOU feel the student has performed? Please circle the description that best describes the performance of this student:
FAIL BORDERLINE
PASS
CLEAR PASS VERY GOOD OUTSTANDING
![Page 33: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/33.jpg)
Borderline Regression Method
Clear Borderline Clear Very Outstanding
Fail Pass Good
Co
mp
ete
nce
ta
sk s
core
10
6.1
0
![Page 34: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/34.jpg)
EXAMINER FORM STICK STUDENT ID LABEL HERE
Check that you have correct
student’s name
and number on the marking sheet.
This station is testing the ability of the student to
Manage a patient with mechanical low back pain Elicit the patient’s expectation of a consultation
COMPETENCE TASK SCORE
Competence task (Maximum mark) Mark ( ½ marks allowed) Comment (only if required)
Student takes a good history including the effect of the
symptoms on the patient’s work (2) 1
Student checks for red flag symptoms: bladder, bowels & saddle
numbness (1 ½) 1
Student explores patients concerns & finds out that patient was
expecting an X-ray; student explains why this is not a good
idea (2 )
½
Student explains natural history of low back pain (1) ½
Student negotiates management plan with patient. This should
using a stronger analgesic drug, encouraging patient to mobilise
and possibility of self certification/ Med3 (2)
1½
Student gives advice on safe lifting (1 ½ ) ½
TOTAL (out of 10) 5
OVERALL IMPRESSION How do YOU feel the student has performed? Please circle the description that best describes the performance of this student:
FAIL BORDERLINE
PASS
CLEAR PASS VERY GOOD OUTSTANDING
![Page 35: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/35.jpg)
Competence Task Score of borderline students at one station
5½ 6 6½ 7
Competence Score out of 10
No. of students
7
6
5
4
3
2
1
Mean = 6.14
![Page 36: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/36.jpg)
Extract of OSCE Spreadsheet showing 10 students’ marks & the pass mark calculated by Borderline
Group Method
StudentStation
A
Station
B
Station
C
Station
D
Station
E
Station
F
Station
G
Station
H
Station
I
Station
J
TOTAL
(out of 100)
a 9.5 5 7 6.5 5.5 6 8 7.5 8.5 7 70.5
b 5 6 8 7.5 2.5 6 6.5 5.5 6.5 9 62.5
c 7.5 7.5 8 6.5 4 7.5 5.5 6.5 6 8 67
d 5.5 6 4.5 5.5 1 6 8 5.5 7 9 58
e 5.5 5 5.5 6.5 5 5 6.5 8 4.5 6 57.5
f 9.5 7 7 5.5 6 6 8 7 6.5 4 66.5
g 9.5 6 7 7.5 8 8 6 6.5 8 8.5 75
h 7 6.5 7 7 6 5.5 5 8 6.5 5.5 64
I 10 5.5 8 8.5 4 9.5 7 6.5 7 7.5 73.5
j 9 7 6 6 5.5 7 8 7 6.5 7 69
Pass
Mark 5.68 6.4 5.81 7 5 6.14 5.82 6.13 6 5.82 59.8
![Page 37: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/37.jpg)
Training package for OSCE examiners
A DVD showing students perform to different levels at same station
![Page 38: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/38.jpg)
Marks from a written exam
10 20 30 40 50 60 70 80 90 100
No. of students
50% Pass mark
![Page 39: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/39.jpg)
“Moderating” (lowering the pass mark to 45%)
10 20 30 40 50 60 70 80 90 100
No. of students
45% Pass mark
![Page 40: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/40.jpg)
Ebel Table
Easy Medium Hard
Essential
Important
Acceptable
![Page 41: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/41.jpg)
Ebel Table
Easy Medium Hard
Essential 90% 80% 70%
Important 60% 50% 40%
Acceptable 30% 20% 10%
![Page 42: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/42.jpg)
Ebel Table
Easy Medium Hard
Essential 1a, 1b 1c 1d
Important 1e
Acceptable
![Page 43: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/43.jpg)
Ebel Table for Extended Matching Questions that have 10 options
Easy Medium Hard
Essential 75% 55% 45%
Important 60% 50% 30%
Acceptable 40% 25% 10%
![Page 44: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/44.jpg)
Ebel Method
Category Number of
Questions in
this category
Proportion of time
minimally competent
student will get this
question correct (%)
Score expected
from minimally
competent
student
Essential Easy 4 75 3.00
Medium 3 55 1.65
Hard 1 45 0.45
Important Easy 4 60 2.40
Medium 5 50 2.50
Hard 2 30 0.6
Acceptable Easy 1 40 0.40
Medium 35 0
Hard 10 0
Total 20 11
![Page 45: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/45.jpg)
Ebel Method
Category Number of
Questions in
this category
Proportion of time
minimally competent
student will get this
question correct (%)
Score expected
from minimally
competent student
Essential Easy 26 75 19.5
Medium 31 55 17.05
Hard 2 45 0.90
Important Easy 32 60 19.20
Medium 33 50 16.5
Hard 11 30 3.30
Acceptable Easy 1 40 0.40
Medium 3 25 0.75
Hard 1 10 0.10
Total 140 77.4
Divided by number of judges (7)
Total 20 11.1
(55.5%)
![Page 46: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/46.jpg)
Summary
Borderline Group Method for OSCE
Ebel Method for written (EMQ or best of 5)
Both
Built on concept of minimally competent student
Use panel of judges to assess each question
![Page 47: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/47.jpg)
The final marks S
tud
en
t
SS
C /20
MC
Q /130
EM
Q/5
0
EM
Q /100
Written
/230
Written a
s
% Wri
tten
/35
OS
CE
as
% OS
CE
/45
To
tal
a 14 79 40 80 159 69.1 24.2 60.7 27.3 65.5
b 17 82 32 64 146 63.5 22.2 52.7 23.7 62.9
c 15.4 69 32 64 133 57.8 20.2 57.2 25.7 61.4
d 12 54 31 62 116 50.4 17.7 46.7 21.0 50.7 Fail
e 14 68 32 64 132 57.4 20.1 47.7 21.5 55.6 Fail
f 18.4 87 41 82 169 73.5 25.7 56.7 25.5 69.6 Merit
g 14.2 91 40 80 171 74.3 26.0 65.2 29.3 69.6 Merit
h 15.6 69 35 70 139 60.4 21.2 54.2 24.4 61.1
I 18.6 79 36 72 151 65.7 23.0 63.7 28.7 70.2 Merit
j 14.2 80 37 74 154 67.0 23.4 59.2 26.6 64.3
![Page 48: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/48.jpg)
HANDLING MARKS
Spreadsheets & Scaling
![Page 49: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/49.jpg)
EXAM BOARDS
![Page 50: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/50.jpg)
GIVING FEEDBACK TO STUDENTS ON THEIR PERFORMANCE
![Page 51: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/51.jpg)
![Page 52: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/52.jpg)
![Page 53: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/53.jpg)
![Page 54: Running an Exam From Beginning to End - University of Bristol](https://reader031.vdocument.in/reader031/viewer/2022021212/62065bfd8c2f7b17300702b2/html5/thumbnails/54.jpg)