external examiners’ briefing day 22 february 2016
DESCRIPTION
Brunel University London Briefing outline Overview of External Examiner role External scrutiny and moderation of assessments Panels and Boards of Examiners Programme and Award Regulations External Examiner annual reports External Examiners’ briefingTRANSCRIPT
External Examiners’ Briefing Day
22 February 2016
Brunel University London External Examiners’ briefing 2016 2
Presenters
Professor Mariann Rand-Weaver, Pro Vice-Chancellor for Quality Assurance and Enhancement
Dr Amanda Harvey, Associate Dean (Quality Assurance), College of Health and Life Sciences
Mr Ruaidhri Donnelly, Head of Quality & Standards Assurance
Plus: Dr Bryan Mase and Dr Jane Lawrie, Associate Deans (Quality Assurance)
Brunel University London External Examiners’ briefing 2016 3
Briefing outline
• Overview of External Examiner role
• External scrutiny and moderation of assessments
• Panels and Boards of Examiners
• Programme and Award Regulations
• External Examiner annual reports
Welcome
Brunel University London External Examiners’ briefing 2016 5
Brunel academic structure
Three large Colleges, each led by a Dean, supported by Vice-Deans Education and Associate Deans Quality Assurance
• Business, Arts and Social Sciences (CBASS)
• Engineering, Design and Physical Sciences (CEDPS)
• Health and Life Sciences (CHLS)
Larger Departments have ‘Divisions’ for different subject area – your appointment as an External Examiner is likely to cover a large slice of a particular Division.
Brunel University London External Examiners’ briefing 2016 6
Current activities and developments
• QAA Review taking place 14-18 March 2016
• New Education strategy and International strategy
• Major Portfolio review now concluding
Brunel University London External Examiners’ briefing 2016 7
Input from External Examiners
Brunel greatly values your work and considers and acts on your reports
We also welcome any feedback about today’s event
Role overview
Brunel University London External Examiners’ briefing 2016 9
Appointment
• Senate Regulation 4 governs the appointment of and role of External Examiners
• At least 1 External Examiner per award bearing Programme
• Appointment normally for 4 years but may be extended by one further year
• External Examiners nominated by Heads of Department and appointed by Senate
Brunel University London External Examiners’ briefing 2016 10
Main functions
• External scrutiny of assessment-setting
• External moderation of summative assessments
• Involvement in Panels and Boards of Examiners which confirm grades and awards
• Other meetings with staff/students as agreed (e.g. project presentations)
• Annual Report (to the VC) on academic standards, assessment processes, student performance, and comparability of overall standards with those of similar programmes in other UK HEIs
Brunel University London External Examiners’ briefing 2016 11
Brunel’s Regulations
All published at www.brunel.ac.uk/about/administration/senate-regulations
• SR2 – Undergraduate programmes/awards (Levels 1, 2, 3)
• SR3 – Postgraduate Taught programmes/awards (Level 5)Note that “levels” are not the same numbers as FHEQ.
• SR4 – Assessment, Boards & Panels of Examiners
Scrutiny and external moderation
Brunel University London External Examiners’ briefing 2016 13
Scrutiny of summative assessment tasksAll summative assessments should be scrutinised by a subgroup designated by the appropriate Panel of Examiners, consulting the External Examiner(s), before final approval by the Panel. [SR4.4]
• External Examiners' views should be sought on all examination papers and other assessments (e.g. proposed coursework titles) at Level 2, 3 and Masters Level, before the papers are printed or other assessments notified to students.
• Assessment criteria, marking schemes, and/or indicative answers should be provided (also see University Grade Descriptors)
• Consider whether the questions: a) are expressed clearly;
b) adequately assess the intended learning outcomes;
c) are in accordance with the overall assessment strategy for the programme;
d) are of an appropriate standard;
e) are associated with appropriate assessment criteria.
• Changes proposed by the External Examiner under should only be rejected with good reason after discussion with him/her.
Brunel University London External Examiners’ briefing 2016 14
External moderation of summative assessments
External Examiners will be provided with [SR4.85]:
• Samples of assessed work (covering full range of student achievement; at least 10% of cohort, or 20% for UG projects / PGT dissertations; all work where class size <10)
• Full set of provisional grades
• All internal moderation or double-marking records (indicating any adjustments already made)
Don’t need to study each piece of work in detail; but consider [SR4.86]:
• Is the work of appropriate standard and comparable with other UK HEIs?
• Do grades/marks assigned reflect the marking criteria, published grade descriptors and intended learning outcomes?
Findings feed into Panel of Examiners consideration/confirmation
Brunel University London External Examiners’ briefing 2016 15
External moderation arrangements
• Samples of student work should be:• Sent prior to the Panel of Examiners (discuss how with the
programme team – e.g. electronically via Blackboard)
• Or, by agreement, available on the morning of the Panel meeting
• You should also be invited to any ‘live’ assessment events, displays, performances etc; or given opportunity to see video recordings (discuss with the programme team)
• You may be asked to moderate placement assessments
All assessed material should be available if you want to see it, either on the day of Panel of Examiners or on request.
Panels of Examiners
Brunel University London 17
Panels and Boards of Examiners – overall role
Common purpose is to ensure:
• Fairness to all students and each student
• Rigour of assessment
• Robustness of the results and decisions
• Maintenance of academic standards
Different types of decision-making:
• Panels are where collective academic judgement is exercised
• Boards confirm that processes have been appropriately carried out, and make decisions on things like reassessment – but extraordinary academic judgements may also be needed
Chairing Boards of Examiners and Panels of Examiners
Brunel University London External Examiners’ briefing 2016 18
Panels of Examiners (PoE) – purpose
Panels of Examiners are where collective academic judgement is employed and EEs have a vital role to play
• Key role: to verify and confirm the marks/grades for each module/block
• Do not look at individual students (unless error or additional info presented)
• May make specific recommendations to Boards concerning particular assessments
See Senate Regulations 4.52 to 4.56
Brunel University London External Examiners’ briefing 2016 19
Issues for Panels of Examiners
• Potential assessment design errors
• Academic judgment
• Moderation effectiveness and issues
• Extraordinary factors/events
• Re-marking/adjustment of results (see next slide)
• Justification of actions
… and ensure recording of decisions at all stages!
Brunel University London External Examiners’ briefing 2016 20
If a Panel identifies a problem
If the Panel is not satisfied with the fairness or integrity of assessments/marking, it should not confirm grades/marks
• Appropriate action to take is normally for the assessors to reconsider grade/marks for all affected students
• Only in very exceptional circumstances may the Panel scale grades/marks for a particular assessment; if so must then record justification and rationale
• Grades/marks may not be adjusted for individual students (unless wrongly recorded, or additional information is presented)
[SR4.55]
Brunel University London External Examiners’ briefing 2016 21
Role of External Examiners at Panels
• Full membership of the Panel of Examiners
• Contribute to the collective academic decision-making
• Have a distinct influence on the PoE (through their independence and their overview)
• EEs are not empowered to change marks or grades – all decisions are made collectively
• May ask to inspect any documents involved in the assessment process
[See SR4.93 – 4.99]
Boards of Examiners
Brunel University London External Examiners’ briefing 2016 23
Boards of Examiners (BoE) – purpose
• To receive confirmed marks/grades from Panels of Examiners
• To consider individual student profile of achievement
• To take into account mitigating circumstances and determine appropriate actions
• To decide on progression and re-assessment
• To recommend appropriate awards for each completed/withdrawn student
• Implement Senate Regulations 4.45-4.51 and 4.57-4.66
• NB Boards now have few powers of discretion within the regulations – but all decisions must still be justified
Brunel University London External Examiners’ briefing 2016
Role of External Examiners at Boards
• Full membership of the BoE
• Contribute to the collective academic decision-making
• General responsibility for all areas covered, not just your own area of specialist focus within this
• Have a distinct influence on the BoE (through independence and overview)
• EEs are not empowered to change marks/grades or progression/classification decisions unilaterally; all decisions are made collectively
• Assure themselves that due process has been followed and appropriate consideration and decisions have been made by the BoE
• See Senate Regulations SR4.93 – 4.99
24
Brunel University London External Examiners’ briefing 2016 25
Vice-Chancellor’s Representative (VCR)
• Award Boards must have a VCR present in order to make awards
• VCR is an experienced member of University professional staff with a good understanding of the Regulations
• VCR has authority to stop the Board should any irregularities come to light, and reports back to the VC on any issues
• VCR (and Assistant VCR, where present) take formal record of BoE decisions – you sign this! – for release to students
Brunel University London External Examiners’ briefing 2016 26
Issues for Boards
• Mitigating circumstances
• Progression and re-assessment
• Awards
• Extraordinary decisions – discretion
• Justification of decisions
… and recording of decisions at all stages!
Brunel University London External Examiners’ briefing 2016 27
Board discretion
Limited scope:
• Action re. Mitigating Circumstances
• Re-assessment timing
• Extraordinary situations
Brunel University London
Mitigating Circumstances (MC Panels)
“A serious or significant unforeseen event, and its consequences, which have significantly impaired the academic performance of a student in one or more assessed activities”
Role of MC Panels (which don’t include EEs) is to:
• Accept or reject MC submissions from students (considering likely impact on studies/assessment, but with no access to academic results/profile in considering cases)
• Make decisions directly re coursework submission deadlines with MCs
• Report to Board of Examiners on all other accepted MCs (identifying serious cases)
See Senate Regulations 4.26 – 4.31, plus further Guidance for Mitigating Circumstances Panels and Boards of Examiners
Chairing Boards of Examiners and Panels of Examiners 28
Brunel University London External Examiners’ briefing 2016 29
BoE actions re. Mitigating Circumstances
• Board paperwork will show where a student’s results have been affected by MCs (as accepted by MCs Panel). Only Chair and Externals should have sight of actual personal details
(which may inform decisions on reassessment timing etc.)
• Actions should minimise disadvantage to students while preserving the integrity of standards and awards. Normal BoE action should be to permit (uncapped) reassessment Need to decide reassessment timing (esp. for severe MCs) Other potential actions: academic decision to assign a grade based on
student’s wider performance/profile; or extraordinary action if agreed with Chair of QAC
• Marks/grades for individual assessment elements (e.g. exam script, coursework essay), as confirmed by PoE, may not be altered.
[See SR4.40-41]
Programme and Award Regulations
Brunel University London External Examiners’ briefing 2016 31
Programme and Award Regulations (SR2 and SR3)
Principles –
• Transparency of standards of awards
• Equity of treatment
• De-modularised Regulations (Assessment Blocks and Study Blocks)
Also note existence of ‘old’ Regulations – may still apply to some students who have previously taken abeyances
Brunel University London External Examiners’ briefing 2016
Brunel’s 17-point grade scale• Grades are the primary
performance measure, as used in award calculations (not marks – though mark equivalence is defined)
• University Grade Descriptors are the primary reference for academic standards
32
Indicative Mark Band
Degree class
equivalentGrade Grade
Point
90 and above 1 A* 1780-89 1 A+ 1673-79 1 A 1570-72 1 A- 1468-69 2.1 B+ 1363-67 2.1 B 1260-62 2.1 B- 1158-59 2.2 C+ 1053-57 2.2 C 950-52 2.2 C- 848-49 3 D+ 743-47 3 D 640-42 3 D- 538-39 Fail E+ 433-37 Fail E 330-32 Fail E- 229 and below Fail F 1
Non-submission Fail NS 0
Brunel University London External Examiners’ briefing 2016 33
Grading and marking at point of assessment
• Departments define whether they are entering marks or grades at element level (by programme)
• SITS (student records database) is set up with details of assessment elements and weightings For Grade entry – SITS assigns a grade point for each element
and calculates weighted average of grade points.
For Mark entry – SITS finds weighted average of marks and assigns corresponding grade point for assessment block
• All calculations and requirements for award are then grade-based
Brunel University London External Examiners’ briefing 2016 34
Progression and Award rules
• Defined minimum grade profile requirements for progression and threshold awards
• “Core” assessments can be defined (must be passed at threshold level)
• Classification determined by simple GPA requirement
• “Borderline” mechanism is strictly deterministic (no BoE discretion)
• No compensation/condonation mechanisms other than strict borderline mechanism
• Standard reassessment rules across University; reassessment volume right = limit
Brunel University London External Examiners’ briefing 2016 35
Core assessments or assessment blocks
• ‘Core’ indicates an assessment block, or element within this, which must be passed with a grade of at least D- (for undergrads) or C- (for postgrads) in order for the student to progress or be awarded
• Core status for specific blocks/assessments must be defined in the programme specification
• All Masters dissertations and all UG Final Year Projects must be designated as core
Brunel University London External Examiners’ briefing 2016 36
Undergraduate Sandwich Placements
• Sandwich students who have passed the placement assessment (minimum grade D-) are eligible for a different award title – normally a “with professional development” suffix
• This award is calculated slightly differently: The placement module grade is included in the Level 2 grade profile for
award.
Percentage contribution of the placement module to Level 2 is defined in the Programme Specification.
• All students who have failed the placement module are eligible for the standard (non-sandwich) award
• Students who did badly in the placement and would get a better classification under the standard award should be offered the choice between them by the Board
[See SR2.42 to 2.44]
Brunel University London External Examiners’ briefing 2016
UG Final Awards – Honours Bachelors
Set out in SR2 Appendices. Based on Level 2 and Level 3 Weighted Grade Profile (1:2 weighting of credit between L2:L3)
• Requirements for Other UG Awards (Integrated Masters, Ordinary, DipHE, CertHE) are also set out in SR2 Appendices
37
Class
Maximum credit
volume of Grade F
Maximum credit volume of
assessment blocks
containing core assessments
below D-
Maximum (non-weighted) credit volume of non-core Grade Band E (E+, E, E-)
Minimum
weighted GPA*
Level 2 Level 3 Level 2 + 3
1st 0 0 0 0 0 14.02.1 0 0 20 0 20 11.02.2 0 0 40 20 40 8.03rd 0 0 40 40 60 NA
Brunel University London External Examiners’ briefing 2016
PGT Final Awards – Master’s
Set out in SR3 Appendices.
• Requirements for PGDip and PGCert Awards are also set out in SR3 Appendices
• If dissertation can be included in the profile for a PGDip award, this must be set out in programme specification
38
Class
Maximum credit volume
of Grades E+, E, E-, F
Maximum credit volume of
assessment blocks containing
core assessments
below C-
Maximum (non-weighted) credit volume of non-
core Grade Band D (D+, D, D-)
Minimum Taught Part Weighted
GPA*
Minimum Dissertation
Grade
Distinction 0 0 0 14.0 A-Merit 0 0 0 11.0 B-Pass 0 0 30 NA C-
Brunel University London External Examiners’ briefing 2016
UG Final Awards – borderline mechanism (PGT very similar)
Set out in SR2 (and SR3 for PGT) Appendices
39
Volume of grades (weighted by assessment block credit value and by Level weighting) in Class or better
Class
1st 2.1 2.2
Minimum Weighted GPA
At least 41% of grades (weighted) in Class or better 13.5 10.5 7.5
At least 45% of grades (weighted) in Class or better 13.0 10.0 7.0
At least 50% of grades (weighted) in Class or better 12.5 9.5 6.5
At least 54% of grades (weighted) in Class or better 12.0 9.0 6.0
At least 58% of grades (weighted) in Class or better 11.5 8.5 NA
At least 62% of grades (weighted) in Class or better 11.0 NA NA
At least 66% of grades (weighted) in Class or better 10.5 NA NA
Brunel University London External Examiners’ briefing 2016 40
Entitlements to reassessment (UG)
• Defined rights to, and limits of, reassessment.
• Not at discretion of Boards of Examiners (unless MCs)
• Entitlement to reassessment where student fails to achieve D- (Level 1, 2, 3) or C- (Level 5) in a block or a core assessment (or fails to achieve a pass in a pass/fail assessment)
• Reassessment volume limit (for each Level) is maximum 60 credits at Level 1, or 40 credits at Level 2, 3 or 5
• Grades for reassessments are capped at D- (Levels 1-3) or C- (Level 5)
Brunel University London External Examiners’ briefing 2016 41
Entitlements to reassessment (PGT)
• Same general principles as for UG
• Reassessment grades capped at C- (threshold grade)
• Entitlement to reassessment in taught part assessments, up to a maximum of 60 credits, where student fails to achieve C- in a block or a core assessment (or fails to achieve a pass in pass/fail assessments)
• Students may submit a dissertation even if they cannot achieve a Master’s degree by doing so
• For dissertation – may revise and resubmit for first-attempt fails between E- and D+; but no resubmission permitted for F-grade dissertation fails or non-submissions
External Examiner Annual Reports
Brunel University London External Examiners’ briefing 2016 43
Report submission
• All External Examiners submit a report annually to the Vice-Chancellor – by 31st July each year
• Request for reports and reminders sent out by the Quality and Standards Office
• Pro-forma/web form provided for the report
• You will be sent a formal response detailing how the University is following up if you raise any concerns
• External Examiners may also write in confidence to the Vice-Chancellor at any time during their period of appointment about any matter relating to their duties
Brunel University London External Examiners’ briefing 2016 44
Response and publication
• College required to respond to the comments made in the External Examiners Report both directly to the External Examiner and in their annual monitoring reports
• University will provide a central response to broader issues, particularly regarding regulations
• External Examiner reports and University responses published to current students and staff. Web page also available to External Examiners.
• Fees paid once EE reports are received
Your questions?
Brunel University London External Examiners’ briefing 2016 46
Further support resources
Guidance and resources about all aspects of these processes are available at
www.brunel.ac.uk/about/quality-assurance/external-examiners
and
www.brunel.ac.uk/about/quality-assurance/assessment
The formal Regulations are published at
www.brunel.ac.uk/about/administration/senate-regulations
Thank you
Please fill out one of the feedback sheets about today’s event