rg170 oxford brookes uni - dera.ioe.ac.uk · the audit process institutional audits are carried out...

Oxford Brookes University

APRIL 2005

ISBN 1 84482 384 9

© Quality Assurance Agency for Higher Education 2005

All QAA's publications are available on our website www.qaa.ac.uk

Printed copies are available from:Linney DirectAdamswayMansfieldNG18 4FN

Tel 01623 450788Fax 01623 450629Email [email protected]

Preface The Quality Assurance Agency for Higher Education (QAA) exists to safeguard the public interest insound standards of higher education (HE) qualifications and to encourage continuous improvement in the management of the quality of HE.To do this QAA carries out reviews of individual HE institutions (universities and colleges of HE). In England and Northern Ireland this process is known as institutional audit. QAA operates similarbut separate processes in Scotland and Wales.

The purpose of institutional audit

The aims of institutional audit are to meet the public interest in knowing that universities andcolleges are:

providing HE, awards and qualifications of an acceptable quality and an appropriate academicstandard, andexercising their legal powers to award degrees in a proper manner.

JudgementsInstitutional audit results in judgements about the institutions being reviewed. Judgements aremade about:

the confidence that can reasonably be placed in the soundness of the institution's present and likelyfuture management of the quality of its programmes and the academic standards of its awards the reliance that can reasonably be placed on the accuracy, integrity, completeness andfrankness of the information that the institution publishes, and about the quality of itsprogrammes and the standards of its awards.

These judgements are expressed as either broad confidence, limited confidence or no confidenceand are accompanied by examples of good practice and recommendations for improvement.

Nationally agreed standardsInstitutional audit uses a set of nationally agreed reference points, known as the 'AcademicInfrastructure', to consider an institution's standards and quality. These are published by QAA andconsist of:

The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ),which include descriptions of different HE qualificationsThe Code of practice for the assurance of academic quality and standards in higher educationsubject benchmark statements, which describe the characteristics of degrees in different subjectsguidelines for preparing programme specifications, which are descriptions of the what is onoffer to students in individual programmes of study. They outline the intended knowledge,skills, understanding and attributes of a student completing that programme. They also givedetails of teaching and assessment methods and link the programme to the FHEQ.

The audit processInstitutional audits are carried out by teams of academics who review the way in which institutionsoversee their academic quality and standards. Because they are evaluating their equals, the processis called 'peer review'. The main elements of institutional audit are:

a preliminary visit by QAA to the institution nine months before the audit visita self-evaluation document submitted by the institution four months before the audit visita written submission by the student representative body, if they have chosen to do so, fourmonths before the audit visita detailed briefing visit to the institution by the audit team five weeks before the audit visitthe audit visit, which lasts five daysthe publication of a report on the audit team's judgements and findings 20 weeks after theaudit visit.

The evidence for the audit In order to obtain the evidence for its judgement, the audit team carries out a number of activities,including:

reviewing the institution's own internal procedures and documents, such as regulations, policystatements, codes of practice, recruitment publications and minutes of relevant meetings, aswell as the self-evaluation document itselfreviewing the written submission from studentsasking questions of relevant stafftalking to students about their experiencesexploring how the institution uses the Academic Infrastructure.

The audit team also gathers evidence by focusing on examples of the institution's internal qualityassurance processes at work using 'audit trails'. These trails may focus on a particular programme orprogrammes offered at that institution, when they are known as a 'discipline audit trail'. In addition,the audit team may focus on a particular theme that runs throughout the institution's managementof its standards and quality. This is known as a 'thematic enquiry'. From 2004, institutions will be required to publish information about the quality and standards of theirprogrammes and awards in a format recommended in document 03/51, Information on quality andstandards in higher education: Final guidance, published by the Higher Education Funding Council forEngland. The audit team reviews progress towards meeting this requirement.

Summary 1Introduction 1Outcome of the audit 1Features of good practice 1Recommendations for action 1Summary outcomes of discipline audit trails 1National reference points 2

Main report 4

Section 1: Introduction: Oxford Brookes University 4

The institution and its mission 4Collaborative provision 6Background information 7The audit process 7Developments since the previous academic quality audit 8

Section 2: The audit investigations:institutional processes 10

The institution's view as expressed in the SED 10The institution's framework for managing quality and standards 10The institution's intentions for theenhancement of quality and standards 12Internal approval, monitoring and review processes 14External participation in internal review processes 16External examiners and their reports 17External reference points 17Programme-level review and accreditation by external agencies 18Student representation at operational and institutional level 19Feedback from students, graduates and employers 20Progression and completion statistics 21Assurance of the quality of teaching staff, appointment, appraisal and reward 22Assurance of the quality of teaching through staff support and development 24

Assurance of the quality of teaching delivered through distributed and distance methods 25Learning support resources 26Academic guidance, support and supervision 27Personal support and guidance 29

Section 3: The audit investigations:discipline audit trails 30

Discipline audit trails 30

Section 4: The audit investigations:published information 40

The students' experience of published information 40Reliability, accuracy and completeness of published information 40

Findings 44The effectiveness of institutional procedures for assuring the quality of programmes 44The effectiveness of institutional procedures for securing the standards of awards 45The effectiveness of institutional procedures for supporting learning 46The outcomes of the discipline audit trails 47The institution's use of the Academic Infrastructure 49The utility of the SED as an illustration of the institution's capacity to reflect upon its own strengths and limitations, and to act on these to enhance quality and standards 49Commentary in the institution's intentions for the enhancement of quality and standards 49The reliability of information 50Features of good practice 50Recommendations for action 51

Appendix 52Oxford Brookes University's response 52to the audit report

Contents

Summary

Introduction

A team of auditors from the Quality AssuranceAgency for Higher Education (QAA) visitedOxford Brookes University (the University) from25 to 29 April 2005 to carry out an institutionalaudit. The purpose of the audit was to providepublic information on the quality of theopportunities available to students and on theacademic standards of awards.

To arrive at its conclusions the audit team spoketo members of staff from across the University,to current students, and read a wide range ofdocuments relating to the way the Universitymanages the academic aspects of its provision.

The words 'academic standards' are used todescribe the level of achievement that a studenthas to reach to gain an award (for example, adegree). It should be at a similar level acrossthe UK.

Academic quality is a way of describing howwell the learning opportunities available tostudents help them to achieve their award. It isabout making sure that appropriate teaching,support, assessment and learning opportunitiesare provided for them.

In institutional audit, both academic standardsand academic quality are reviewed.

Outcome of the audit

As a result of its investigations, the audit team'sview of the University is that:

broad confidence can be placed in thesoundness of the University's currentmanagement of the academic quality ofits programmes, and

broad confidence can be placed in theinstitutional level capacity to manageeffectively the security of its awards.

Features of good practice

The audit team identified the following areas asbeing good practice:

the quality of support for postgraduateresearch students

the on-line Personal Information Portaldeveloped by the University for itsstudents

the level of accessibility of staff withinschools and their support for students,underpinned by the information providedto staff by Student Services

the series of themed audits undertaken bythe Academic Policy and Quality Unit.

Recommendations for action

It would be advisable for the University to:

strengthen the quality assurance processesat institutional level in order to secure asufficiently effective oversight by AcademicBoard and its committees of theiroperation in the schools

continue to address the identifieddeficiencies in the quality of learningexperienced by some students in the firstyear of semesterisation

develop and publish a plan for thesemesterised academic year which ensuresoptimal coordination and operation

review its assessment procedures withparticular attention to consistency, timingand load on students.

It would be desirable for the University to:

work more closely with the officers of theStudents' Union in order to improve theutility of student involvement at theinstitutional level

continue to develop a more strategicapproach to the use and analysis ofstatistical data within review and decision-making processes

make more effective use of the annualreview process and develop further itsformal systems for the dissemination ofgood practice across the institution.

Summary outcomes of the disciplineaudit trails

Business and management: BA (Hons)Business and Management; BA (Hons)International Business Management; BA (Hons)

Institutional Audit Report: summary

page 1

Business Logistics; BA (Hons) BusinessInnovation and Enterprise; MSc Business andEnterprise

English studies: BA Single Honours, EnglishStudies; BA Combined Honours; EnglishStudies; Postgraduate Certificate/PostgraduateDiploma/MA, Modern and ContemporaryPoetry

Construction management and urbandesign: BSc (Hons) Building; BSc (Hons)Construction Management; MA Urban Design

Mechanical engineering: BEng (Hons)Mechanical Engineering; BEng (Hons)Automotive Engineering

Philosophy and religious studies: BACombined Honours in Philosophy; BACombined Honours in Religious Studies

The audit team also looked at these five specificareas of provision by undertaking disciplineaudit trails to find out how well the University'ssystems and procedures were working at thediscipline level. The University provided theteam with documents, including studentassessed work and, here too, the team metwith staff and students. The findings of theteam in the five discipline areas supported theoverall confidence statements given above. Theteam considered that the standard of studentachievement in each of the five discipline areaswas appropriate to the title of the award and itslocation within The framework for highereducation qualifications in England, Wales andNorthern Ireland, published by QAA, and thatthe quality of learning opportunities available tostudents was suitable for a programme of studyleading to that award.

National reference points

To provide further evidence to support itsfindings the audit team also investigated theuse made by the University of the AcademicInfrastructure which QAA has developed onbehalf of the whole of UK higher education.The Academic Infrastructure is a set ofnationally agreed reference points that help todefine both good practice and academicstandards. The findings of the audit suggest

that the University has embedded thesedevelopments into its management of qualityand standards in a timely manner.

From the end of 2004 QAA's audit teams willcomment on the reliability of the informationabout academic quality and standards thatinstitutions will be required to publish, which islisted in the Higher Education Funding Councilfor England's document 03/51, Information onquality and standards in higher education: Finalguidance. The institutional audit process hasincluded a check on the reliability of theinformation sets published by institutions in theformat recommended in HEFCE 03/51. TheUniversity is alert to the publicationrequirements and the audit found that it hadmade substantial progress towards fulfilling itsresponsibilities in this regard.


page 2

Main report

Main report 1 An institutional audit of Oxford BrookesUniversity (the University) was undertakenduring the week 25 to 29 April 2005. Thepurpose of the audit was to provide publicinformation on the quality of the University'sprogrammes of study and on the discharge ofits responsibility for its awards.

2 The audit was carried out using a processdeveloped by the Quality Assurance Agency forHigher Education (QAA) in partnership with theHigher Education Funding Council for England(HEFCE), the Standing Conference of Principals(SCOP) and Universities UK (UUK), and whichhas been endorsed by the Department forEducation and Skills. For institutions in England,it replaces the previous processes ofcontinuation audit, undertaken by QAA at therequest of UUK and SCOP, and universal subjectreview, undertaken by QAA on behalf of HEFCE,as part of the latter's statutory responsibility forassessing the quality of education that it funds.

3 The audit checked the effectiveness of theUniversity's procedures for establishing andmaintaining the standards of its academicawards; for reviewing and enhancing the qualityof the programmes of study leading to thoseawards; and for publishing reliable information.As part of the audit process, according toprotocols agreed with HEFCE, SCOP and UUK,the audit included consideration of examples ofinstitutional processes at work at the level of theprogramme, through discipline audit trails(DATs), together with examples of thoseprocesses operating at the level of the institutionas a whole. The scope of the audit encompassedall of the University's provision leading to itsawards, excepting its collaborative arrangementswhich are to be considered in a separate audit.

Section 1: Introduction: OxfordBrookes University

The institution and its mission

4 The University has its origins in the OxfordSchool of Art which was founded in 1865.

Subsequent development of technical andother subject areas led eventually to itsdesignation in 1956 as a College of Technology,at which time it consolidated on theHeadington Campus site. The College becameOxford Polytechnic in 1970 and, in 1975, theinstitution amalgamated with Lady Spencer-Churchill teacher training college, acquiring asecond major campus at Wheatley. OxfordBrookes University was incorporated in 1992taking its name from John Brookes, the Vice-Principal from 1928 and Principal from 1934until 1956. There was further expansionthrough a merger with Westminster College in2000 and the establishment of a third majorcampus at the College's site at Harcourt Hill. InSeptember 2004 a fourth major site at MarstonRoad became operational and enabled theconsolidation of the School of Health and SocialCare at the new facility. The University has bothtaught and research degree-awarding powers.

5 Total student numbers were reported as17,940 at the time of the July 2004 submissionof data to Higher Education Statistics Agency(HESA), of whom 68 per cent were full-timeand sandwich and 32 per cent were part-time.Undergraduates constituted 74 per cent of thestudent population and 70 per cent of allstudents were aged 21 or over at that time.International students and non-UK EU studentsaccounted respectively for 13.4 and 5 per centof the student population.

6 In 2000 senior management of theUniversity initiated a number of changes to theorganisation of the institution. The academicstructure for the University is now based uponeight large schools instead of the previous 13smaller ones: the Business School; the School ofArts and Humanities; the School of Biologicaland Molecular Sciences; the School of Healthand Social Care; the School of Social Sciencesand Law; the School of Technology; the Schoolof the Built Environment; and the WestminsterInstitute of Education. In terms of studentnumbers the largest schools are the BusinessSchool, the Westminster Institute of Educationand the School of Health and Social Care. TheUniversity has a broad portfolio of provision


page 4

that is weighted towards professional subjectareas, for example nursing, business, law,architecture, engineering, computing andeducation. The reduction in the number ofschools created larger organisational units witha higher level of devolution, the University'sintention being to allow them to be better ableto manage their responsibilities such as financialcontrol and quality assurance. Also completedin 1999-2000, a restructuring of support areasled to the formation of six directorates with keyfunctions relating to the student experience,these being delivered in particular by theDirectorate of Learning Resources and theDirectorate of Academic and Student Affairs.

7 Schools and directorates are cost centreswith responsibility for devolved budgets.Budgetary control and direction is achievedthrough an annual strategic planning roundinvolving the Senior Management Team (SMT)and senior staff within the schools anddirectorates. The planning process also coversthe allocation of student numbers to schoolsand identification of new programme provision.School management structures vary from schoolto school but include support for the dean ofschool from a number of senior staff, includingassistant deans and school administrators. Thelatter generally manage the day-to-dayoperational activities of the school with theassistant deans usually undertaking specific rolesand responsibilities, for example, undergraduateor postgraduate programme provision; research;resource allocation. Decisions regardingacademic provision within the subject areas aregenerally devolved to schools. Schools can takedecisions to develop or close individualprogrammes but decisions to close subject areasare taken by senior management and are basedon consultation with appropriate management,staff and student groups.

8 In 2002-03 there was a review ofmanagement and governance which resulted ina reduction in the number of committeesreporting to Academic Board. The formal inputof schools and directorates into strategicdecision-making in the University takes placeprimarily through the Executive Board (EB)

which determines the University's strategic andoperational plans. This body, constituted inJanuary 2004, comprises all deans and directorstogether with the members of the SMT.Decisions regarding financial management ofthe University are taken by the Vice-Chancellorand the Board of Governors and in these areasEB acts in an advisory capacity.

9 At the time of the QAA audit in 2001, theUniversity was carrying out a wide-rangingstrategic review. Led by the SMT, the major aimwas 'to achieve efficiencies through a deeperunderstanding of the organisation and itsenvironment, and to re-cast the University in away which gives the institution the best possiblechance to thrive, academically and financially,during the next ten years'. The decision to moveto a semester structure for the academic yearhad been taken, but many other components ofthe review had not been completed at the timeof the 2001 audit. The key review areas, whichwere considered initially by four strategicworking groups, included:

revisiting the University's Mission andStrategic Plan

reviewing the University's undergraduateand postgraduate course portfolio

changing the structure of the academicyear

revising the Undergraduate ModularProgramme (UMP)

reviewing the quality and customer careprocesses

redesigning the University's managementinformation systems

revising the financial and strategicplanning model.

10 Following university-wide consultation in2003 a revised 'Mission for Oxford BrookesUniversity' was developed:

'Oxford Brookes University will contribute to theintellectual, social and economic developmentof the communities it serves through teaching,research and enterprise of the highest standards'.

Institutional Audit Report: main report

page 5

Implementation of the mission is providedthrough three strategic goals which were set outin the 2003 Academic Plan. This plan is astatement of the University's intention to be: apremier learning and teaching institution; aresearch community that values equally researchand knowledge transfer; and a regional partnercontributing to social and economicdevelopment. In order to deliver the mission andstrategic goals a number of key objectives wereidentified (and updated in 2004):

develop a distinctive and sustainableacademic portfolio of the highest quality

develop research excellence in allacademic schools

increase the range and volume ofknowledge transfer from the University'sresearch activities and expertise

secure a leading role in the social, economicand cultural development of the region

increase the diversity of the student bodyto represent a wider range ofbackgrounds, cultures and countries

develop the quality and increase thediversity of staff

be financially self-sustaining

increase the quality and efficient use ofuniversity facilities.

In order to deliver the mission and strategic goalsthe University has developed a range of targetedoperating strategies. Of these, the strategicstatements which have a direct relationship tothe academic context include the Learning,Teaching and Assessment Strategy (LTAS); theResearch and Knowledge Transfer Strategy; theStrategy for Regional Development; the StrategicStatement on Quality and Customer Care; theInformation Processes and Systems Strategy; thee-Learning Strategy; the CommunicationsStrategy; and the International Strategy.

11 The University has introduced a newstrategic planning process to align the objectivesof schools and directorates with the University'sobjectives as outlined above. Progress inachieving these objectives is monitored through

performance indicators, with reporting to EB. Tosupport this process, EB has identified around 40indicators. These include data which have beenprovided for many years (such as studentapplications, intakes and targets) together withnewly developed indicators which reflectemerging central and regional governmentobjectives (for example, percentage of graduatesfinding employment within the region). Regularreporting on these indicators began in December2004. The introduction of a further 20 indicatorswas deferred by EB in early February 2004because no data to support them were availableat that time. A number of these related tostudent satisfaction and, following theUniversity's first large-scale, cross-institutionalsurvey in 2004, these data are now available(see paragraph 80 below).

12 A major element of the strategic reviewinvolved a detailed re-examination of theUniversity's course portfolio and researchactivities. After developing a number of criteriafor assessing the strengths and weaknesses ofprogrammes and subject areas a major reviewof the course portfolio was undertaken. Initiallythe programmes within the UMP were reviewedand this resulted in decisions to phase outcertain subject areas where ongoing difficultiesaround student recruitment affected the longer-term viability of programmes (for example,geology, cartography and civil engineering),and a commitment to develop other areas.Subsequently programmes outside the UMP,and particularly those in the postgraduateportfolio, were similarly evaluated in terms oftheir recruitment, viability, and potential futuredevelopment. The University has recentlydeveloped a number of Foundation Degrees.They share a common structure and a Guide tothe Development of Foundation Degrees hasbeen produced to aid development teams. Allthe Foundation Degrees contain work-basedlearning as a component.

Collaborative provision

13 The self-evaluation document (SED)highlighted a recent wide-ranging review of theUniversity's processes for the management and


page 6

quality assurance of its collaborativepartnerships. In view, however, of the size andcomplexity of the University's collaborativeprovision, it will be the subject of a separate,future, audit. Therefore it does not form part ofthe current institutional audit or of this report.

Background information

14 The published information available forthis audit included:

the report of the QAA continuation auditof the University in October andNovember 2001 which was published inMarch 2002

quality assessment reports and subjectreview reports produced since 1995

the reports of QAA developmentalengagements covering the provision inarchitecture, music (both summer 2003),and history (summer 2004)

the information available on University'swebsite.

15 The University initially provided QAA with:

an institutional SED, with links to extensive documentation on a CD-ROMand on the University's website. Thisdocumentation included details of theUniversity's organisational and committeestructures, the academic regulations and other regulatory documentation; arange of policy and strategy statements;and the main quality assurance proceduresof the University

five discipline self-evaluation documents(DSEDs) for the areas selected for DATs, together with the relevantprogramme specifications

access to the University's intranet.

16 During the briefing and audit visits, theaudit team was able to examine a range of theUniversity's internal documents, both in hardcopy and through the intranet facility. Duringthe audit visit the University provided the teamwith a range of additional documentationrelating to the selected DATs, including a fullrange of examples of students' assessed work.

The University provided a number ofsubsequently identified and requesteddocuments during the visit and the team alsohad full access to the University's intranet andto the Personal Information Portal (PIP). Theteam appreciated the unrestricted access that itwas given to these sources of information.

The audit process

17 Following a preliminary meeting at theUniversity in September 2004 between the QAAAssistant Director and representatives of theUniversity and students, it was confirmed thatfive DATs would be conducted during the auditvisit. On the basis of the SED and other publishedinformation, the audit team confirmed that theDATs would focus on taught programmes inbusiness and management; English studies;construction management and urban design;mechanical and automotive engineering; andphilosophy and religious studies. The Universityprovided QAA with DSEDs in March 2005.

18 A briefing visit was conducted at theUniversity on 14 to 16 March 2005. Thepurpose of this visit was to explore with theVice-Chancellor, senior members of staff of theUniversity and student representatives mattersof institutional-level management of quality andstandards raised by the University's SED, thestudents' written submission (SWS), and thepublished documentation. At the close of thebriefing visit, a programme of meetings for theaudit visit was agreed with the University.

19 At the preliminary meeting for the audit,the students of the University were invited,through their Students' Union (SU), to submit awritten submission expressing views on thestudent experience at the University, andidentifying any matters of concern orcommendation with respect to the quality ofprogrammes and the academic standards ofawards. They were also invited to give theirviews on the level of representation afforded tothem, and the extent to which their views werenoted and acted upon. In generating theirwritten submission, the SU established aworking party to oversee the production of theSWS, which included an external consultant


page 7

from another University. This group organisedfocus group meetings with students to designthe questionnaire, field tested it and issued iton-line to all Oxford Brookes students. Thequestionnaire was also made available bycanvassers at the SU building and at theUniversity's sites. In total, nearly 1,700responses were received, representing 10.2 percent of the student population. The SWScontained statistical analysis and textualcomment on five main areas: the quality andreliability of the information published by theUniversity; academic performance (includingassessment); the experience of students aslearners; student participation in quality andstandards; and the experience of differentcategories of student. Following the analysis ofthis survey, the SU submitted the document tothe Academic Board (AB) meeting in November2004. The meeting of AB concluded that theSWS was an unbalanced document and the SUofficers were requested to submit a shortcovering paper putting the conclusions intocontext. The SWS was subsequently submittedto QAA in January 2004. The audit team isgrateful to the SU for preparing this documentwhich it found to be a thorough and carefullyconsidered report. The SWS provided the teamwith helpful information relevant to theexperience and views of the student body, andthe team considered that it constituted anexcellent model among similar studentcontributions to the process of institutionalaudit (see also paragraph 78 below).

20 The audit visit took place from 25 to 29April 2005. Eight meetings were held during thevisit with groups of staff and students from theUniversity. Meetings were also held with staffand students in the five subject areas selected forthe DATs. The audit team comprised Professor JBeeby, Professor R Craik, Mr M Hill, Ms A JKettle, Mr B Robinson, Dr P Smith, auditors, andMrs R Goggin, audit secretary. The audit wascoordinated for QAA by Dr P J A Findlay,Assistant Director, Reviews Group.

Developments since the previousacademic quality audit

21 The University was last audited in Octoberand November 2001, with the audit reportpublished in March 2002. The AB of theUniversity received the report of the continuationaudit at a meeting in May 2002. In June 2002,the AB received the University's response whichwas presented in a summary table indicating theproposed action plan in response to the report'srecommendations. It was agreed that the Qualityand Standards Committee (QSC) would haveoperational responsibility for monitoring thevarious strands of the response. Clear lines ofresponsibility, timescales and responsibilitieswere indicated. The most recent update of theaction plan, dated November 2004, indicatedthat of the 18 actions required all had eitherbeen implemented or given carefulconsideration by the University. The onlyremaining area where work was ongoing wasthe action to provide opportunity for greatersocial interaction across the postgraduatecommunity. The audit team reviewed theaction plan and came to the view that theUniversity had acted appropriately to addressmany of the issues raised in the 2001 audit.

22 The 2001 audit highlighted the goodpractice of centralised student support and theguidance given to staff in performing theirstudent support functions. It also praised theinvolvement of student representatives oncommittees and the responsive process for theinduction of new undergraduates.

23 The audit report containedrecommendations related to ensuring that thequality and standards loops between centralcommittees and schools are always closed. TheSED stated that the annual review andreporting process is now the vehicle by whichthis requirement was now addressed by theUniversity. In view of comments raised insubject review reports at that time, the 2001audit also identified the need for there to begreater consistency in assessment proceduresand practices across schools. In the SED, theUniversity drew the attention to the redesign of


page 8

all programmes at the time of the introductionof the new semester-based academic yearwhich had provided the opportunity todemonstrate this consistency and to makeacademic progression explicit. Consistency isalso supported by the University LTAs whichrequires that the learning outcomes of allprogrammes are linked to assessment.

24 The report of the 2001 audit also referredto the need to ensure that focused staffdevelopment supported the outcome of thestrategic review. As part of the 2002-04 HumanResources Strategy the University introduced anew unified personal review scheme for all staff.The first round of this scheme now beencompleted and its monitoring by the Universitysuggests that it has been well implemented(see paragraph 89 below).

25 The report also identified a number ofareas that the University was advised toconsider. These included ensuring that thecomplex change agenda on which it hadembarked was managed effectively. Many ofthe developments planned in 2001 had sincebeen followed through by the University,although work continued on revising theUniversity's financial and strategic planningmodel. The audit team noted, however, thatsince the publication of the 2001 report thepace of change in the University had, in manyrespects, increased and the University nowfaced new challenges. At the time of the 2001audit the University was carrying out a wide-ranging strategic review. Since then, theUniversity has introduced a new strategicplanning process, moved to a semesterstructure, reviewed the UMP and postgraduateprogrammes portfolio, reviewed the quality andcustomer care processes, redesigned themanagement information systems and madechanges to its committee structures.

26 The most challenging change for theUniversity, in the view of the audit team, hasbeen the move to a semester structure, and theSED acknowledged the demands on theinstitution which had arisen from thisdevelopment, and the concurrent concerns of

students and staff. In May 2001 the ABapproved the move to a semester structure andimplementation was scheduled for September2004. Preparation for the change was led bythe Semesters Project Office (SPO) which wasestablished in September 2002 with the ProVice-Chancellor as Project Director. A projectManager was seconded from the Academic andPolicy Quality Unit (APQU). The SPO alsoestablished five working groups to handlespecific aspects of the implementation (theLearning & Teaching Working Group, theStudent Support Working Group, the Access,Recruitment & Publicity Working Group andthe Systems & Resources Working Group). Theprincipal potential benefits arising from therestructuring of the academic year were seen asto provide substantially more time during thesummer for staff to conduct research andengage in knowledge transfer activities; and toprovide the opportunity for the University topromote 'summer University' activities. Inaddition the move from three terms to twosemesters reduced the number of formalsummative assessment points from three to twoand, therefore, reduced the burden of formalassessment. The University's UMP andpostgraduate programmes were revised and re-approved during this process and wide-rangingamendments were made to qualitymanagement systems. At the AB meeting inMay 2001 SU officers and studentrepresentatives, in accord with the results of anSU referendum, expressed opposition to themove to semesters. While continuing to opposethe move to semesters, the SU has cooperatedwith the SPO to make the transition as smoothas possible. Meetings with students have beenheld regularly and student concerns havehelped to shape the structure of the newacademic year and assessment. The SED set outclearly the issues relating to the introduction ofsemesters which had been identified by theUniversity's own systems and by its externalconsultants; the audit team found that thesecontinued to have a considerable impact on thestudent experience in the University (seeparagraphs 44 to 49).


page 9

Section 2: The auditinvestigations: institutionalprocesses

The institution''s view as expressed inthe SED

27 The SED set out in detail the aims whichthe University seeks to accomplish through itsframework for managing quality and standardsin order to fulfill the commitment in its missionto provide teaching of the highest standards.The policy is to ensure that 'there is anappropriate degree of ownership of qualityassurance mechanisms at each level ofoperation'. The University further seeks 'toestablish, maintain and enhance qualitythrough a series of rigorous procedures'. Thesenior university committees concerned withquality and standards have delegated to theschools powers to operate their ownprocedures. It is intended that schools can'operate their quality management processeswith a degree of flexibility' but that in this theyare constrained by the general operationalprinciples set by the University in the Qualityand Standards Handbook (QSH). For example,the QSH sets out the requirements for annualand periodic review, for the approval of newand revised programmes and the expectationsfor professional and statutory body (PSB) visitsand accreditation.

28 In order that the central committees canmaintain oversight of school qualitymanagement activity, members of the APQUare assigned to schools as Quality AssuranceOfficers (QAOs). Their role includes both adviceto school staff and the monitoring of schoolactivities. The audit team confirmed the closerelationship between the QAOs and the schoolsand their detailed involvement with the entirerange of school processes. In addition to theirmonitoring, communication and advice roles,the QAOs are able to identify good practiceand to disseminate it. QAOs also work closelywith the Head of Quality Assurance (HQA) onthe development of quality assurance processesand academic policy.

29 Schools are required to develop their ownprocedures for the management of quality andstandards. These are subsequently put beforethe QSC (see below paragraph 33) for approvaland review. This is intended to ensure that theytake proper account of university and externalquality assurance principles. The SED explainedthat use by the schools of their own proceduresfor programme development and approvalenables academic staff responsible forprogrammes to assume ownership of therelevant quality issues at an early stage.

The institution's framework formanaging quality and standards

30 Following a review in 2003, in whichconsideration of 'the policies and management ofquality within the University' formed an element,a new committee structure has been adopted.Only a single layer of committees now reports tothe AB. Those committees concerned with qualityand standards are QSC, Learning and Teaching(LTC), Research and Knowledge Transfer (RKTC)and the eight school boards. In a separatedevelopment, the APQU was restructured and anew HQA appointed.

31 The AB is the senior academic committeeof the University and has overall responsibilityfor the academic standards and quality of allprogrammes. It oversees admissions,assessment and awards and is required tosatisfy itself by receipt of minutes and reviewprocesses that any powers delegated to otherbodies are correctly applied. Both the Boarditself and its committees must include studentmembers. The current membership of theBoard includes elected staff representatives andeight student members. The University hasadopted the convention that committeeminutes are circulated to AB memberselectronically and not usually discussed, so thatthe Board can focus its meetings on specificdiscussion issues.

32 Matters of strategy and planning are dealtwith by the EB which has deans, directors andthe SMT as members and acts in an advisorycapacity to the Vice-Chancellor. The


page 10

establishment of the EB, also following the2003 review, was not, at the time, withoutopposition and, indeed, AB voted against itsformation. The review consultation paper statedthat 'the Academic Board and committees willbe consulted on decisions about academicactivities'. The audit team noted the fine linewhich a group such as the EB must drawbetween its overall executive role, and theformal responsibility for academic matterswhich must rest with the AB. The team wouldencourage the University to pay carefulattention to ensuring that the terms ofreference of the AB are fully observed.

33 The SED identified the QSC as 'the body inwhich quality assurance policy and procedure isbrought together and determined'. It hasrepresentation from each school and from theuniversity executive structure and somedirectorates. Its remit includes the assurance,maintenance and enhancement of quality andstandards and the quality of student supportactivities. It is expected to develop policies andprocedures and to monitor their implementation.It also has oversight of the monitoring and reviewof academic programmes. The schoolrepresentatives on QSC are designated qualityassurance contacts and are members of theQuality Forum (QF) (see below paragraph 38).According to the SED 'they have been identifiedas the key conduits for communication betweenthe school and the committee'.

34 The LTC is responsible for policy andimplementation in respect of learning, teachingand assessment. Its remit specifically covers e-learning and student support, guidance andretention. The membership is largely composedof senior teaching and student support staff. TheSED recognised past difficulties in thecommunication of LTC decisions to teaching staffand noted that 'the committee is investigatinghow to improve the channels of communication'.

35 The remit of the RKTC includes 'researchstudents and the quality, standards andoperation of research programmes and degrees'.There is also a Research Degrees Committee(RDC) with delegated powers to award research

degrees. It reports to RKTC and also directly toAB in relation to research degree registrationsand the assessment and conferment of researchdegrees. The RDC works closely with theGraduate Office and through disciplinesubcommittees. The audit team learned duringits visit that approval has now been given forthe establishment of a Graduate School.

36 School boards play a major part in themanagement of academic quality andstandards. Although they are required tomanage quality and standards processes as setout in the QSH, they have freedom todetermine their own committee structure,subject to minimum terms of reference for theboard, and have flexibility on detailedprocedures. The audit team noted that theresulting variability in the management ofquality and standards within schools placedadditional responsibility on the higher-levelcommittees to ensure that institutional policiesare correctly and effectively followed.

37 Each school is required to publish and tooperate its own policy and procedures for theassessment of students' work. These covermatters such as double marking and moderationof summative assessed work. The University hasin place guidance on the procedures forexamination committees and makes themavailable to external examiners through theexternal examiner resource web pages. Theassessment load for each module is consideredas part of the approval process and forms part ofthe programme specifications. In addition,programme committees are required to haveregard for assessment scheduling and volume.

38 Three times each year the HQA, theQAOs, the Director of Academic and StudentAffairs and the quality assurance contacts fromeach school meet as the QF. The meetings areinformal, although notes are kept, and allowdiscussion of a wide range of issues concernedwith quality, including consideration of areaswhere new or revised policy may be required.The SED noted that the QF also provides anopportunity for the sharing of good practiceacross schools. The audit team noted the


page 11

spread of topics discussed in recent years, suchas external examining, assessment, externalreviews and personal development planning,and concluded that the QF was a significantcontributor to quality assurance within theinstitution and a valuable initiative.

39 Considering the overall framework forquality assurance, the audit team found thatthe chain of delegation of quality assurancematters from the AB to QSC to schools, (withthe APQU sometimes as an intermediary,preparing summary reports), was a long one.The team considered that this, together withthe degree of flexibility and variability allowedto schools, the number of committeesreporting to AB, and the Board's approach tothe consideration of committee minutes, hadresulted in a system in which the Board'soversight of quality assurance processes takingplace at the school, programme or modulelevel was relatively weak. While recognising therole played by QSC in this regard, the teamnevertheless advised that the communication ofquality matters to the AB and theirconsideration by that body needs to bestrengthened if it is to carry out effectively theresponsibilities which rest with it.

The institution''s intentions for theenhancement of quality and standards

40 According to the SED, the University 'iscommitted to enhancing not just the quality ofall its academic programmes, but the wholestudent experience'. A paper from the APQU inJune 2004 set out a Strategy for QualityEnhancement of Academic Programmes. Oneof the more important features of this strategyis the intention to improve the sharing of goodpractice, whether derived from internal orexternal reviews or from considerations at theprogramme level. The University has developedwhat is described in the SED as a coherentapproach to enhancement, involving a newcentral post, improved learning, teaching andassessment practice, staff development,enhanced student support, improved facilitiesand better dissemination of good practice.

41 A significant new focus for developing thequality of academic programmes has beenprovided by the appointment of a Head ofLearning and Teaching who works with the LTCon the development and implementation of theUniversity's LTAs. The LTC maintains acontinuing overview of progress in meeting thevarious objectives of the strategy, in addition toconsidering other proposals for developmentsin learning and teaching. Developments inlearning and teaching practice are also soughtas a means for enhancing the qualityexperienced by students. Poor attendance atearlier learning and teaching forums has led toa new approach exemplified by a recentconference on 'Delivering Inclusivity andDiversity'. A new peer reviewed e-Journal ofLearning and Teaching has been establishedand joins a formerly paper-based publication,Teaching News, which has been an e-journalsince 2003. These are expected to providewide-dissemination of pedagogicdevelopments. The SED particularly stressed theemphasis placed on staff development (seeparagraphs 96 to 103) as a contributingprocess for quality enhancement.

42 Close attention is also paid to studentsupport (see paragraphs 119 to 124), beginningat the pre-enrolment stage and proceedingthrough the provision, during the course, ofhandbooks and personal and academic guidanceto careers guidance. The implementation ofpersonal development planning (PDP) forundergraduates is a further extension of thisactivity. In addition, a Quality and CustomerCare policy has been developed as part of theStrategic Review. This is intended to enhance,among other things, the service provided tostudents. The objectives include extension ofannual and periodic review processes intosupport directorates, improved understanding ofstudent satisfaction and progress oncommunications and information processes.

43 The SED stated that the reapproval processassociated with the move to semesters hadoffered opportunities for enhancement. It hadinvolved a redesign of the academic programmeswhich allowed the definition of a closer


page 12

relationship between the learning, teaching andassessment strategies and the learning outcomesof the programmes. Students were intended tohave 'a better understanding of the academicstandards and levels of achievement expected'from clearer assessment criteria relevant to eachlearning outcome. Definitive programmespecifications support this process. Although theaudit team recognised that the procedures forthe redesign and approval of programmes hadbeen carefully managed, it was too early for theteam to judge the extent to which the overallaims had been achieved.

44 Much staff effort at all levels of theUniversity during the three years preceding theaudit had been occupied with theimplementation of the decision to move from anacademic year consisting of three terms to one oftwo semesters (see paragraph 26 above). Thenew teaching and assessment year lasts for 28weeks (22 of them devoted to teaching), endingjust after the middle of May (a shorter periodthan previously). The transition took effect fromthe start of the current academic year, 2004-05,at which time all three years of study werechanged. The audit team noted that this was anextremely challenging change for staff andstudents alike, especially since there have beenassociated course and timetable changes (seebelow). The process of change was managed bythe Semesters Project Board (SPB) which beganmeeting in November 2002 and was supportedby the Themed Working Groups. The SPB hasreported to the AB on a regular basis. The teamwas aware that its visit coincided with theUniversity's first opportunity for an overall reviewof the change and consideration of anydifficulties. Following an SPB meeting in February,a detailed report on the implementation ofsemesters was received and discussed by the AB.Although generally positive, the report drewattention to a number of problems which havearisen and which were also noted by the teamduring discussions with staff and students.

45 Among the potential problems identifiedearly by the SPB is the structure of theacademic year. Despite being appreciated atthe beginning of the process and carefully

considered, the attempt to squeeze the entirefirst semester into the period before Christmashas led to difficulties with examinations andsome disruption to the final week of teaching.The calendar for the academic year 2005-06 isalready fixed so that it is difficult to resolve thisproblem fully in the short term. A further issueis the management of assessments, concerningwhich students complained of bunching ofcoursework and examinations. In some cases,assessment loads may have increased withsemesterisation, despite the University's wishand intention to reduce them.

46 The thorough review and reapprovalprocess undertaken as part of the semesterisationproject was also intended to reduce the overallnumber of modules taught. This has been done,but it has combined with the changes in thetimetable structure to cause difficulties for somestudents, particularly those studying part-time, infollowing their intended programme. Transitionalarrangements have been put in place to minimisethe disruption to courses, but affected studentshave to discuss their options individually withacademic staff. The audit team neverthelessfound that these difficulties persisted, and theteam met, mid way through the secondsemester, some students who did not know howor whether they would be able to complete theirintended course. A further consequence for somestudents of the move to semesters was broughtto the attention of the team and had also beenidentified by the SPB. It concerned difficultiesover the timetabling of contact hours, referred toin the University as 'slotting'. Affected students,both full and part-time, have experiencedchanges such as bunching of contact hours oralterations to evening sessions which they havefound difficult to manage. A related issue whichstudents brought to the attention of the team isthe timing of the reading weeks which, they felt,had in this year been less coordinated and henceless useful than previously.

47 In the student meetings during several ofthe DATs the audit team identified concerns fromstudents about the implementation of thesemester structure. These related to theperceived heavier workload, particularly in the


page 13

first semester, the resultant effect on timetablingduring the week, the impact on part-timestudents and the assessment load. Staff atvarious levels of the University recognised thechallenge that semesterisation had presented.The SPB has reported regularly on theimplementation of the process and at the March2005 meeting of the AB an interim report wasconsidered. This catalogued the difficulties beingexperienced by students under four mainheadings: learning and teaching, assessment,systems and resources (which includes thesemester calendar) and student support. The SPBreport recommended that these issues shouldprimarily be handled through annual reviewreports, noting that most of the changes need tobe implemented at programme and modulelevel. The Board decided that the SPB shouldcontinue in existence to monitor theimplementation of semesters in conjunction withthe LTC and the QSC. A full review ofsemesterisation was to be carried out at the endof the first full academic year of operation. Whilethe team acknowledged that many of theactions would need to be addressed at a locallevel, it considered that the University would alsobenefit from putting in place a plan and atimetable against which progress could bemonitored by the committees.

48 The SU had also recently undertaken asurvey on semesterisation which indicated that amajority of the student body felt thatsemesterisation had not been a positive move.The officers stated that some students hadsuffered from a lack of choice of modules as aresult of semesterisation and that there had beena lack of consistent advice on the managementof the modules that could be taken. In the worstcases this had resulted in unbalanced workloadson students in the two semesters.

49 The audit team found that the University isconcerned about the enhancement of itsprogrammes and has put in place a range ofappropriate processes to support this. However,despite the considerable efforts at all levels, themajor programme of change on which it hasbeen working since 2001 has not been withoutproblems and it was clear to the team that a

number of students, including some of thesignificant cohort of part-time students, hadbeen adversely affected. The team concludedthat many members of the student body hadexperienced one or more of the problemsidentified above during the first year ofsemesterisation and it was likely that the qualityof their learning experience had been reduced inconsequence. It is not only important for theUniversity to eliminate or reduce these problemsfor future years, but also to pay continuingattention to those who have experienceddifficulties with the progress of their studiesduring the current year, so as to ensure that thequality of their overall course of study is notdiminished. The team recognised that theUniversity had recently made progress withidentifying and considering the problemsoccasioned by the move to semesters. Itnevertheless advises the University to maintain itsefforts to deal with the problems as a matter ofurgency and particularly to seek to minimise anydislocations to the studies of current students.

Internal approval, monitoring andreview processes

50 The University's requirements and processesfor the approval of new and revised programmesare set out in great detail in the QSH and aredescribed as 'robust' in the SED. Consideration ofnew programme proposals is essentially a two-stage process, resource and policy matters beingdealt with first either through the school'splanning agreement and the University'sprogramme development calendar or throughthe EB. In the Business School, the audit teamnoted particularly well-developed procedures forestablishing the business case for a newprogramme proposal. Once it has been agreedthat a proposal satisfies the planningrequirements, it proceeds to developmentthrough a programme development team (PDT).This team has the task of establishing the detailsof the programme and preparing the necessarydocumentation, including programmespecifications and the proposed studenthandbook. Other considerations, such as likelytake-up and the profile of graduates, must alsobe addressed. The proposal must then be placed


page 14

before a University approval panel chaired by adean and usually having external members. Thispanel considers whether the proposal satisfies alarge number of criteria set out in the QSH and, ifsatisfied, makes a recommendation to the QSC.Any conditions set by the approval panel must bemet by the PDT before the link QAO forwardsthe required certification form to the QSC.

51 Major revisions to existing programmesmust follow essentially the same procedure. Thecriteria for treating a revision as major are set outin the QSH and include, as noted in the SED'proposals which affect learning outcomes, theaddition of new awards, new modes of delivery,significant changes to assessment strategy andchanges to title'. Minor revisions are subject toapproval by the dean and by the link QAO andusually require consultation with externalexaminers. The University's Academic Regulationsrequire that formal consultation with studentsalready on the course must take place before anychanges can be implemented. Reapproval panelswhich operated to approve revised programmesas part of the semesterisation project followed thesame process. External panel members were usedif there was a significant change in theprogramme, but for the most part this was notjudged to be necessary and advice was soughtonly from external examiners.

52 The SED described annual programmereview (APR) as 'a core element in theUniversity's processes'. The review is conductedby the programme team and considers externalexaminer reports, progression data and studentevaluation among the evidence. The QSHdescribes in detail the purposes, conduct andintentions of the review process. The report ofthe review, using a standard template, mustinclude an action plan for the following yearand programme teams are 'asked to identifyexamples of good practice or innovation forwider consideration'. The programme reportsare considered at a school annual reviewmeeting which is expected both to reflectcritically on 'issues arising from the previousacademic year' and to set 'key priorities for theforthcoming year'.

53 The annual review reports from schoolsare then forwarded to the APQU and are notshared directly or through University committeepapers with other schools. The SED explainedthat in the APQU issues arising from the schoolannual review reports are 'collated andaddressed to areas, such as directorates, whereaction is requested'. This has sometimes provedto be a slow process as matters haveprogressed through the committee structure.Following a recent review, an enhanced processhas been introduced 'with a clearer focus onidentifying major issues and obtaining directresponses by interaction between the schoolsand directorates concerned'. The audit teamconsidered that, given the significant effortrequired to produce the reports and therelevance of their content, it would also bebeneficial for them to be seen by staff in otherschools. Although the APQU prepares asummary report for the QSC on the outcomesof the school reviews, the process is not, in thejudgement of the team, being used to itsgreatest advantage in terms of securingconsistency of the student experience acrossthe institution. Similarly, although discussion atthe annual school review meeting provides theopportunity for consideration of common issuesand sharing good practice within schools, thatcan only happen between schools through themedium of the APQU. This interventioninevitably reduces the effectiveness of theprocess below that which would be achieved ifthe annual review reports were also discussedat meetings involving school academic staff.

54 The University uses a periodic reviewprocess on a six-year cycle. In a change from anearlier pattern which used schools as the basisfor review, the process 'is now focused at thelevel of the subject or discipline', normallycollecting together all programmes within aparticular JACS code. The SED argued that 'thischange has allowed the University to focus moreeffectively on academic issues'. As set out in theQSH, each school is responsible for reviewingthe programmes within its portfolio. The QSHmakes explicit reference to the relevant sectionof the Code of practice for the assurance of


page 15

academic quality and standards in highereducation (Code of practice), published by QAA,and lists the questions that the review shouldaddress. The panel will normally be chaired bythe dean of school or a nominatedrepresentative and a standard membership isprescribed. The panel receives a range ofdocumentation including APR reports andexternal examiners' reports. It meets formally fora one-day event during which it meets with staffand students. At the end of the meeting theprogramme teams receive an oral report on thepanel's findings. The written report is consideredby the school board and forwarded to theAPQU together with the subsequent actionplan. Finally, the report and the response areconsidered and monitored by QSC. Both QSCand LTC receive annual collations of issues andgood practice identified in the periodic review.The audit team considered that, if carried out asspecified in the QSH, this constituted aneffective process of periodic review.

55 The University operates a second reviewprocess referred to as themed audits. The topicsfor these audits are approved by QSC and thereviews are undertaken by APQU, looking at 'themechanisms for assuring quality across particularaspects of the University's work'. Among theprocesses that have been audited arepostgraduate research programmes, academicappeals and complaints and delegated validationprocedures. The audit reports are received by theQSC and appropriate school-level committees foraction on the recommendations. The audit teamread some of the reports and found them toreflect a careful evaluation of the relevantprocesses coupled with useful recommendationsfor improvements. The example of the audit ofpostgraduate research programmes (seeparagraph 116 below) further showed this to bean effective mechanism for the enhancement ofquality processes. The team commends thethemed audit process as good practice.

External participation in internalreview processes

56 The SED stated that the University assuresitself that new programmes are of theappropriate academic standard for the awardbeing proposed by using an approval processthat requires the involvement of externaladvisers with appropriate subject knowledgeand expertise. Approval panels are specificallyrequired to ensure that external advice has beensought and responded to during thedevelopment and the SED gave a number ofexamples of the influence of PSBs andemployers on the development of new orrevised programmes. The level of externalinvolvement as set out in the QSH is that'external advisers must be included in PDTdiscussions wherever possible'. The QSH doesnot specify that there must be externalmembers of either the PDT or the approvalpanel, except when the approval process islinked to PSB approval, in which case a nomineemay be included in the approval panel.However, the audit team was told by staff thatthere usually is an external member on approvalpanels unless the new programme is very similarto an existing programme. This suggests thatthe very detailed statement on the use ofexternal advisers appended to the QSH could beusefully updated to reflect current practice.

57 It is university policy that each periodicreview panel should contain an externalmember who can be either a PSB representativeor a subject specialist. The specialist couldrepresent an employer's view or provide anacademic input. It is expected that the externalpanel member 'will take a full and active part inthe review process, including attending theevent itself'. Review reports seen by the auditteam confirmed that effective use is made ofexternal panel members in the review process.

58 The University's arrangements for externalparticipation in programme approval andperiodic review were judged to be appropriateand working well in practice. The audit teamfound that the University has sound policieswhich are implemented in schools to ensure


page 16

appropriate and productive externalparticipation in its quality assurance processes.

External examiners and their reports

59 The SED confirmed that it is a Universityrequirement that external examiners areappointed to each award-bearing programme.Their role is to monitor the programme'sassessment processes and consider the overallacademic standards. Schools are responsible fornominating external examiners and nominationsare finally approved by the QSC on behalf ofAcademic Board. The University normallyappoints external examiners for four academicyears. On appointment APQU provides initialinduction to new external examiners through itsweb pages. Most external examiners haveexpressed satisfaction with this arrangementand the information provided, but some haveasked for written guidance, and this is beingdeveloped. The schools provide individualbriefing for the examiners about the subjectarea and the programmes to be covered. Thisinformation includes school or departmentalmoderation policies and programmehandbooks. Examiners are asked to comment intheir annual report on the adequacy of thebriefing information received.

60 External examiners are required to attendsubject examination boards and to submit anannual report, for which the template hadrecently been extensively revised. Reports arereceived by APQU and circulated to schools. Thedean of school is required to ensure the externalexaminer receives a formal response and schoolshave adopted various procedures to ensure thatexternal examiners are informed about theoutcomes of their reports. Examiners are alsoasked to comment on whether they believe thatprevious recommendations have been addressedto their satisfaction and this is monitored by theQAOs. The external examiner reports aresynoptically summarised in an annual report bythe QAOs and this overview is considered atUniversity level by QSC on behalf of AcademicBoard. The SED pointed out that the recentmove to semesterisation had utilised the externalexaminer system to ensure maintenance of

quality and standards and the relationship withexternal examiners in some cases went beyondthe scrutiny of standards to involve activeguidance in the enhancement of provision.

61 DATs found evidence of externalexaminers engaging well with the examinationand quality enhancement processes. Externalexaminers within the DATs provided full anddetailed reports and it was noted that thetemplate for these had been adjusted in orderto accommodate the requirements forpublished teaching quality information (TQI).Staff in the DATs confirmed that externalexaminers were notified of actions taken inresponse to their reports, primarily throughreceiving the annual review reports, althoughthe audit team noted that considerablevariability in the process of reporting actionsexisted across the institution.

62 It was stated in the SED that theUniversity's policies and procedures meet theprecepts of the Code of practice, Section 4:External examiners. Overall, the audit team wasable to confirm this and was satisfied that theUniversity had established an effective externalexaminer system which reflected recent sector-level developments. Although practice acrossthe schools is variable in some respects, theteam was convinced that the relationship withexternal examiners provided satisfactoryassurance of standards.

External reference points

63 The SED stated that the University hasengaged actively from the outset with all theelements of the Academic Infrastructure,published by QAA.

The Code of practice64 The University has taken an active approachto the use of the various elements of the Code ofpractice. The University has reviewed its practicein light of the precepts of each section of theCode and identified areas where practice alreadymeets the expectations expressed in the Codeand those where additional work is required.Relevant sections of the Code are used as a


page 17

starting point for the themed audits (seeparagraph 55 above). In general the Universitywas confident that the Code was effectivelymapped and integrated into the QSH, and auditteam was able to confirm this.

The FHEQ65 The SED stated that the Universitywelcomed the development of The framework forhigher education qualifications in England, Walesand Northern Ireland (FHEQ) and that it expectsthe qualification descriptors to be used as a partof the approval and review process for academicprogrammes. In relation to the FHEQ all of itsawards have been reviewed and are consideredto be in line with the relevant level-descriptors. Itwas noted that a particular feature of theUniversity's undergraduate awards was thedistinction between level 1 modules, defined asstage 1, and level 2 and 3 modules which weredefined as stage 2; however, clear distinction hasbeen made between the levels in order tocomply with the FHEQ.

Subject benchmark statements66 In its SED the University emphasised thatsubject benchmark statements have beenintegrated into the consideration of theappropriateness of the learning outcomes andcurriculum of both new and existing programmes;during the redesign of all programmes for theintroduction of semesters, teams were requiredto demonstrate how they had taken account ofthe relevant benchmark statement as well asthe relevant qualification descriptor.

67 Evidence from the DATs showed thatsubject benchmarks had often been a positiveand useful reference point for curriculum design.Evidence from the DATs also demonstrated astrong awareness of subject benchmarks butnoted that a number of vocational areas placedmore emphasis on professional, statutory andregulatory body (PSRB) expectations. PSRBs arean important external reference point for manycourses, exercising their own influence oncurriculum design.

Programme specifications68 All programmes offered by the Universityhave a programme specification, the developmentof which is a part of the approval process.Programme specifications were extensivelyreviewed as a part of the redesign ofprogrammes. The University is now movingtowards publishing programme specificationson its website. The University's LTAs recognisesthe value of programme specifications and usedthe redesign process as a mechanism to linkmore closely the learning, teaching andassessment strategies and the learningoutcomes of programmes in a constructivealignment. Evidence from the DATs confirmedthat programme specifications were in place forall programmes and were being used effectivelyfor curriculum management and for studentinformation. Specifications seen by the audit teamshowed appropriate references to the elementsof the Academic Infrastructure, with good useof subject benchmarks. They also includedreference to the University's strategies forlearning, teaching and research. The Universityis using programme specifications effectively.

69 Overall, the audit team concluded that theUniversity had a positive and effective approachto the recognition in all of its programmes ofthe external reference points provided by theAcademic Infrastructure, and these have beenappropriately adopted and embedded as a partof its internal procedures.

Programme-level review andaccreditation by external agencies

70 The University underwent two subjectreviews following the 2001 audit, since whenthere have been a number of changes in theparticipating programmes and schools inresponse to the recommendations made. Morerecently, three developmental engagementshave taken place. The reports were consideredin the departments, the schools and by QSC,which also received an overview paper of issuesarising from them.


page 18

71 The University's portfolio of programmesincludes many professional areas, leading tonumerous visits and meetings concerning PSRBapproval or accreditation. Staff in departmentswork closely with their professional colleaguesand schools seek to combine accreditation visitswith university review processes when possible.The visits are managed by the schools but withuniversity support available. The audit teamnoted the close and responsive interactionsbetween the University and the PSRBs. On theevidence available to it, the team judged thatthe University was making appropriate use ofexternal reports to inform its management ofquality and standards.

Student representation at operationaland institutional level

72 The SED stated that students have anumber of formal and informal routes forrepresentation within the institution. TheUniversity's Student Charter sets out the rightsand responsibilities of student representatives. Atthe operational level, all programmes haveelected student representatives and their role isdefined in the relevant programme handbooks.Student representatives are members ofprogramme committees and in some schoolsthey sit on the appropriate school anddepartmental committees. The SU has a studentrepresentative coordinator to liaise with therepresentatives and produces a useful booklet onbecoming a student representative. The SU runstraining programmes for student representativesin their role on programme committees.

73 Notwithstanding concerns expressed inthe SWS relating to student representation andfeedback (see below paragraph 78), the auditteam found that students met during the auditconfirmed the general effectiveness of the studentrepresentation system at programme level andidentified positive changes that had beenachieved following their comments in feedback.The students' representatives at programmelevel contribute to the annual review meetingsfor each programme. They are also invited tobecome members of the panels carrying outperiodic review although take up so far has been

low. For example, in the context of the periodicreview in the School of the Built Environment,staff had asked the SU for assistance in thenomination of student representatives.

74 In the SWS, the SU commented that theunderstanding among students of the role ofstudent representatives was low and stated thatthere was room for improvement. It waspointed out that the training and supportprovided by the SU can only be made availableto those representatives that they know aboutand that the communication of this informationfrom programmes could be improved. The auditteam heard from the Director of Academic andStudent Affairs that the University wasinvestigating the possibility of working with theSU to identify ways to increase studentinvolvement and provide specific training forthem in the periodic review process. Officers ofthe SU stated that they would welcome greaterconsistency in the student representation systemat school level. Through the DATs the teamrecognised the variation that existed in thesystem of student representation at school level.

75 At institutional level, elected members ofthe SU are represented on the AB, on itssubcommittees and on the Board of Governors.The SED commented that apart from the ABand the RDC, the attendance by students hadbeen limited and that the University intendedto work with the SU to improve this. The SEDstated that the Vice-Chancellor and the DeputyVice-Chancellor and Registrar have regularlyscheduled meetings with the SU officers whichthe audit team learnt were six-weekly in thecase of the Deputy Vice-Chancellor andRegistrar. The SU officers expressed the viewthat, even with these meetings, they did notfeel that issues which they raised were beinggiven sufficient attention by the seniormanagement of the University. Following thepublication of the SWS and its subsequentconsideration by the AB, the SU officers felt,according to their discussion with the team,that they had no longer been able to representthe student body in an appropriately receptivecontext; the officers were conscious of the needfor both sides to work to improve this situation.


page 19

76 The SED stated that evidence from annualand periodic reviews suggests that moststudents feel that their views are taken intoaccount, and the evidence gathered by theaudit team in its meetings with students in theDATs tended to support this view. However, thiscontrasted with the concerns expressed to theteam by the SU officers, and with the findings ofthe SWS survey. The team formed the view thatstudent representation was generally effective atthe level of the programme committees.Students are represented and able to make theirviews heard on matters of immediate concernto them. The University is conscious of the needto encourage the involvement of studentrepresentatives and is working in a number ofways to do this. At the institutional levelstudents' views are articulated through the SUrepresentatives who sit on all the UniversityBoards and committees, with the exception ofthe EB. The officers of the SU work hard togather and report the views of the studentbody, but at the time of the audit they did notbelieve that there was a receptive response totheir contributions. The team encourages theUniversity and the SU to work to reinforce thisimportant relationship at the institutional level.

Feedback from students, graduatesand employers

77 The SED claimed that the University placesgreat importance on collecting and actingupon feedback from students and that themechanisms are effective. At the module level,feedback is collected using evaluation forms orquestionnaires. At programme and school levelthe inputs of students' representatives are themain mechanism. The questionnaires used atthe module level contribute to the productionof the annual evaluation reports of modulesalthough the methods used are variable. Theviews of students are also raised and discussedat programme committee meetings. Studentsmet by the audit team confirmed that theyprovided regular feedback on their modulesusing the module evaluation questionnaires(MEQs). They also pointed to the informalchannels of communication with programme

and module leaders that usually led to thespeedy resolution of any issues.

78 The SWS produced by the SU summarisedthe results of a carefully conducted survey of thestudent body, with a response rate of just over10 per cent (see also paragraph 19 above). Theconclusions reached in the survey were: (mostimportantly) that a substantial majority of therespondents (73 per cent) were happy or veryhappy with their experience at the University;that there was some concern regarding theaccuracy of information, in particular pre-arrivalinformation; that some students wereconcerned about aspects of group-work; thatthere was a substantial amount of dissatisfactionabout the availability of resources; that studentrepresentation was not perceived as alwaysworking effectively; that many students felt thattheir feedback was not listened to, or had notseen the result of it; and that students fromnon-traditional backgrounds or postgraduatestudents were least satisfied with theirexperience at the University. In the course of theaudit, the audit team was shown an action planproduced by the University which was intendedto address the issues of concern that had beenidentified in the SWS.

79 The SED recognised that at times feedbackhas not been as effective as it might be,confirming the view expressed in the SWS. Thesurvey undertaken prior to the writing of theSWS revealed that approximately one third ofstudents felt that developments in their courseswere not communicated to them and only justover one third felt that their comments made infeedback were taken seriously. The SWS hadcommented that the situation could beimproved if the good practice in some schoolsof the University was to be spread more widelyacross the University. The audit team noted thatin the University's response to the SWS therewere no specific actions identified in relation tothe issue of students feeling that theircomments were not taken seriously, althoughthe HQA and the QF had been assignedresponsibility for addressing this.


page 20

80 During 2004 the University undertook aninstitution-wide student satisfaction surveywhich achieved a 35 per cent response rate.This revealed that there was satisfaction withthe learning and teaching experience and theorganisation of courses and expressed anumber of particular areas as very satisfactoryincluding the timing of classes, theencouragement from staff, the flexibility of themodular system and the availability and qualityof lecture notes and handbooks. The only issuethat was rated as unsatisfactory within theperception of learning and teaching was thecost of course materials and this had beenreferred to the Value for Money Group.Concerns were also raised about the availabilityof multiple copies for books at the library sites,and as a result the University has made anadditional grant of £60,000 to the library to bespent on multiple copies of books. The resultsof this survey have been translated into anaction plan which indicates the issues, theactions and the responsibility for action. TheUniversity plans to repeat the studentsatisfaction survey on a regular basis.

81 With respect to feedback from graduatesthe SED acknowledged that the University hadbeen less successful. The pattern of collection offeedback is variable, with active alumniassociations in, for instance, the subject areas ofarchitecture and hospitality, leisure and tourism.In the Department of Accounting the survey ofgraduates in 2002 revealed that they wantedgreater contact with employers and this has ledto a number of visits from representatives of theprofessional bodies. The Business School haslaunched Bacchus, a dedicated website for allits alumni and current students. Exit surveys areundertaken in the Business School to gather theviews of students on their programmes but thisis not universal practice. Overall the gatheringof the views of graduates was judged in theSED to be ad-hoc, and the evidence available tothe audit team suggested that this was anaccurate assessment.

82 The views of employers contribute to thedevelopment of curricula in some schools, mostnotably, those with PSRB links. For example the

School of Technology and the Departments ofPlanning and Real Estate and Construction haveprogramme advisory boards. The School ofMechanical Engineering has reintroduced anindustrial advisory board to meet once eachsemester to advise on course content. TheUniversity actively seeks the views of employersat validation and periodic reviews and the auditteam noted that the periodic review of BuiltEnvironment programmes had includedappropriate representation of externals. Otherfeedback on programmes is obtained throughthe liaison between the staff of the Universityand placement employers. These inform thecurrency of the curriculum and the preparationof students for employment.

83 Overall, the audit team formed the viewthat there were appropriate mechanisms inplace to gather the views of students at themodule and programme level. The teamwelcomes the decision by the University tocontinue to undertake the student satisfactionsurvey on a regular basis and notes thedemonstrated commitment to acting on theoutcomes of the survey, guided by action plans.In this context, the team noted with interestthe different outcomes, with regard to someaspects of the student experience, of thesurveys conducted by the University and theSU. The University may wish to reflect, throughsuch a comparison, on the best approach toidentifying students' concerns. With respect toemployers' and graduates' views the Universityhas itself recognised, and the team supports theview, that much could be learnt from theinitiatives in different schools to ensure effectivegathering of feedback.

Progression and completion statistics

84 The SED outlined ways in which theUniversity uses progression and completionstatistics. These are one benchmark of studentperformance and they are given thoroughconsideration in examination committees.Standard module report forms are used inmany schools; these include a statistical analysisof results and they explicitly compare resultsyear on year. While a range of data was


page 21

provided for the annual review report it wasnoted that the only University requirement wasan analysis of degree awards, with identificationof any trends which may be developing inrelation to classes of awards, mark averages etc.It was also noted that in the annual reviewprocess programme teams generally did notprovide any commentary on the statisticalinformation and data provided. The teamconsidered that this was an area which wouldbe enhanced by a more detailed analysis of allthe statistical information provided.

85 The use of progression and completionstatistics for quality assurance purposes at aninstitutional level is one where work is ongoing.The University has used a course and studentmanagement system for many years to recordboth the details of the course structure and ofall the programmes within the UMP andstudent performance and progression withinthe modules and programmes. The system isnow being replaced by a new student recordsmanagement system (eCSIS). This is expectedto enable a more sophisticated analyses ofstatistical information, including that relating tofirst destination statistics and informationrelating to widening participation initiatives.

86 Periodic review panels are provided withstatistical evidence on student progression,achievement and employment for the periodunder review, although not all panels considerthis information in detail. Evidence from theDATs indicated that no detailed analysis of datawas included in the periodic review reports. Withregard to the University's performance, the auditteam found that the data provided from theindividual DAT areas indicated good progressionand completion rates, and HESA data madeavailable to the team indicated a satisfactoryperformance against specific indicators andshowed that the University is generallyperforming at its benchmark level with regard toprogression and completion. At the institutionallevel, the team also saw an example of a reportto the Board of Governors which providedvarious statistics related to finance, postgraduateresearch and external funding, however, no dataor analysis for overall University performance in

progression and completion were providedbeyond the normal HESA return.

87 The audit team found that there wasextensive use of data at the local level in thecontext of annual review and in the assessmentprocess. Student data had also been used inanalytical studies by various committeesdeveloping policy, for example, withdrawal rateswere considered by the LTC. However, the teamfound little evidence that data analysis wasbeing used systematically to inform strategicpolicy-making within the academic committeestructure. The University recognised theimportance of the provision and analysis ofstudent data and was moving towardsenhanced methods of capture, analysis, and themore comprehensive review of statisticalinformation. However, the data collectedcentrally does not appear to be used effectivelyto inform or manage programmes at school andcourse level. Overall, the data gathering systemsappeared to be effective and had the potentialto provide detailed data at institutional level.The University will wish to consider, however,whether its current procedures enable it tomake the best use of the available data.

Assurance of the quality of teachingstaff, appointment, appraisal andreward

88 The SED stated that the University hasadopted a comprehensive recruitment andselection policy to ensure that the mostappropriate candidate is selected, and that itspolicy on equality of employment opportunityis followed. All posts have documented jobdescriptions and person specifications that aredetermined by the school or directorate, toreflect the nature and content of the role.Schools are responsible for the recruitment ofhourly-paid staff who are then issued withformal contracts of employment by theUniversity. The Directorate of Human Resourceshas taken over the responsibility for advertisingsuch posts, while schools will continue to carryout the selection. It is intended that this willenable more effective monitoring of therecruitment and selection process. For all posts,


page 22

members of appointment panels must haveattended the University's recruitment andselection training course; the content of thiscourse was reviewed and updated during the2003-04 academic year. The reasons forselection decisions are recorded to ensure thatthe process is transparent. References are takenup and qualifications are verified.

89 As part of the 2002-04 Human ResourcesStrategy, a new Personal and ProfessionalDevelopment and Review process (PPDR) for allstaff was introduced from June/July 2003. Thenew process covers the assessment ofperformance and achievement of objectivesover the previous year, career progression anddevelopment needs, and objectives for thefollowing year. The PPDR scheme applies to allstaff, including fractional posts. For academic staff,appraisal is usually by the line manager, who willhave been trained as a part of the process. Theaudit team was able to confirm that the systemwas well embedded and staff who met the teamspoke of appraisal positively as allowing for theidentification of development needs.

90 The schools and directorates provide mostof the elements of the induction process inaccordance with the guidelines provided by theDirectorate of Human Resources. These provide aset of organisational and job-related informationwhich is designed to supplement thedocumentary information and welcome packthat is provided by the Directorate of HumanResources with the contract of employment. At approximately six-week intervals, theDirectorate runs an introduction day for newly-appointed staff, in which seniormanagers introduce participants to the day-to-day operations of the University, and a'marketplace' is offered through which serviceproviders can provide more information aboutthe University's facilities. A member of theUniversity's SMT attends to make a shortpresentation about the University's Mission andto answer questions. The content and format ofthis introduction day was reviewed andrestructured in 2003. Separate sessions ondisplay screen equipment and manual handlingare offered to all staff for whom they would be

appropriate. Some of the schools anddirectorates have their own handbooks and/ormanuals and may also organise school inductiondays to supplement the University-wide material.

91 With effect from the autumn semester2004, new academic staff will participate in aformal three-year programme of support thatwill incorporate mentoring on their researchactivities, research training, undertaking theUniversity's Postgraduate Certificate in Teachingin Higher Education (PGCTHE), and access to a£3,000 bursary to support work on a researchfunding application.

92 The criteria for promotion to senior andprincipal lecturer are set out in the employmenthandbook. At both levels, effectiveness andinnovation as a teacher, researcher and scholar,administrator, and in consultancy and externalactivity, can be advanced in support of apromotion. All candidates are expected todemonstrate teaching effectiveness andexcellence. The criteria and procedures forpromotion to professorial status were revised in2002, and include as one of the three thresholdcriteria, the 'record and reputation of thecandidate as a teacher, including theircontribution to the development of the subjectand to course development'.

93 A key element of the University's LTAs is to'encourage and reward teaching excellence andto establish teaching and course delivery basedon evidence of effective practice'. Thepromotion procedures for principal lecturer andprofessor grades already incorporate a set ofcriteria reflecting contribution to learning andteaching, and from 2004-05 candidates forpromotion will be expected to submit aportfolio of evidence of their contribution insupport of their application. In 2000, as part ofthis strategy, two initiatives were launched toenhance learning and teaching and disseminategood practice. One initiative led to the creationof designated principal lecturer posts in allschools, specialising in teaching and learning;the other established 12 academic fellowshipsfor one-year projects in learning and teachingsubmitted under a university-wide competitivebidding procedure. In both cases, a compulsory


page 23

criterion for appointment was a demonstrablerecord of excellence in learning and teaching.

94 In addition, and as part of the HumanResource Strategy, six two-year teachingfellowships have been awarded annually since2003 to reward teaching excellence and tosupport pedagogic development in areas thatcontribute to the University's strategicobjectives. These fellowships have beensupplemented from 2004 by two-year associateteaching fellowships for less experienced staffto support their further development in areas oflearning and teaching, as appropriate to theirschools. Additionally, two two-year learningsupport fellowships will be offered for staff inposts which directly support the developmentof learning and teaching excellence.

95 On the basis of the published informationand from its discussions with staff, the auditteam considered that the arrangements forappointing and appraising staff were effectiveand clear. The team was satisfied that theUniversity had effective systems in place toencourage staff in the pursuit of excellence inresearch and that the more recentdevelopments and initiatives had alsoestablished learning and teaching as an area ofsignificant career development.

Assurance of the quality of teachingthrough staff support anddevelopment

96 The University's PPDR process includes therequirement that schools and directoratesmaintain staff development plans. These arebased on the development needs of allmembers of staff which are identified annuallyas part of the PPDR process, and are alsorelated to key university and school objectives.Hourly-paid staff are not expected to undergoPPDR, although they do have access to all staffdevelopment activities provided by the University.

97 A central feature of the new framework forstaff development is the three-year rolling staffdevelopment plan which is produced by eachschool and directorate. It is revised annuallyand links corporate and school/directorate

objectives to development priorities. The plansform the basis for determining allocations fromthe central staff development fund and are alsolinked to school/directorate human resourcestrategies. Each plan incorporates staffdevelopment supporting the delivery ofresearch, learning and teaching, andorganisational objectives, together withcontinuing professional development and thetechnical and management/administrativerequirements of the school's operation. Plansare geared to the achievement of the keyresults areas for the school or directorateconcerned over the next planning period, andcomprises both specialist provision for its ownspecific needs and components of thecorporate employee development portfoliodelivered by the Oxford Centre for StaffLearning and Development (OCSLD), withinthe Directorate of Human Resources.

98 For all new teaching staff, half-time andabove, with less than five years teachingexperience and more than a one-year contract,it is compulsory to complete the PGCTHE. Thisis a one-year credit-bearing master's levelcourse, accredited by the Staff Education andDevelopment Association, in collaboration withOxford Brookes, and by the Institute forLearning and Teaching in Higher Education(ILTHE). Schools receive part-time hourscompensation to cover the attendance of newstaff on the course. Staff from OCSLD evaluatethe course annually and consult deans andother academic colleagues on the form andcontent of the course. Additionally, OCSLDprovides a 10-session Associate Teachers'Course for staff and postgraduate researchstudents who undertake more than 50 hours ofteaching per annum. Accreditation is optionaland each year a proportion of the participantsseek credit by completing the assessment.Those who undertake less than 50 hours ofteaching per year are also entitled to attendindividual sessions of interest to them. Both ofthese courses were accredited by ILTHE, and areopen to other staff for whom it is notcompulsory. For more experienced staff, aPostgraduate Diploma in Learning and


page 24

Teaching in Higher Education is available, andthis can lead to an MA in Education. Allparticipants on the PGCTHE are required tohave a mentor appointed in their school. Thereis also a new university mentoring schemewhich offers the opportunity for all staff, at alllevels, to be mentored.

99 In addition, OCSLD offers an annualprogramme of staff development workshops,including topics on aspects of learning andteaching. In the area of e-learning, theUniversity has an e-Learning Strategy and thisarticulates with the Human Resources Strategyto support staff development in this area.Training, support and advice is provided jointlyby the Media Workshop and OCSLD who alsoprovide a programme of staff developmentsessions on e-learning. The Media Workshopalso hosts the Learning Technologists' Forumthat provides additional support and maintainsthe Brookes Virtual website, which includes achecklist of quality measures for tutors andschools to check before going live.

100 Each school and directorate has adesignated OCSLD link consultant who worksclosely with the dean/director and staff to definethe plan and to determine how training needsare identified and met. They will arrange forOCSLD to provide in-house development advice,consultancy and workshops on current learningand teaching issues identified by the school.

101 Another element of the Human ResourcesStrategy is the Senior Staff DevelopmentProgramme. The Programme provides workshopsand seminars for senior staff with managerialresponsibilities across the University and alsoidentifies and addresses their individualdevelopmental needs. For staff aspiring tobecome senior managers OCSLD runs a CoreManagement and Leadership Programme whichis externally assessed and leads to an NVQ award.

102 All schools are required to have peer-observation schemes in operation. The exactmechanisms of the schemes vary betweenschools, but they are all expected to beconsistent with the principles described in theStrategy for Quality Enhancement of Academic

Programmes. LTC is planning to evaluate theeffectiveness of these schemes during 2004-05.The audit team particularly noted the innovativeaspects of the peer observation scheme (PELT)operated by the Westminster Institute forEducation (WIE) (see below paragraph 185).

103 The audit team in its meetings with staffin the DATs and in the institutional meetings,and through its reading of the appropriatedocuments, was able to confirm that the StaffDevelopment Policy was being realised in theschools and directorates in line with Universitypolicies. The team came to the conclusion thatthe University gave high priority to staff supportand development as a driving factor in thepursuit of excellence in teaching. Both new andestablished staff expressed support andenthusiasm for the variety of trainingopportunities available to them. Particularpraise was directed at the coordinating role ofthe OCSLD.

Assurance of the quality of teachingdelivered through distributed anddistance methods

104 The University has developed a wide range of collaborative partnerships rangingfrom large overseas partnerships to smallspecialist colleges. The responsibility formanaging these partnerships at a strategic levelis the responsibility for the LearningPartnerships Advisory Group reporting to theSMT through its chair, the Deputy Vice-Chancellor (Academic). The quality assurancearrangements for the provision at these partnerinstitutions is to be the subject of a separateQAA collaborative provision audit in the futureand so lies outside the scope of this report.

105 The University has been developingextensive e-learning capacity and at the time ofthe audit had nearly 10,000 students makinguse of e-learning in their studies. Thisdevelopment is supported by a comprehensivee-learning strategy. The majority of this capacityfor flexible and distance learning (FDL) is forthe support of students for whom e-learning isone part of their learning within the on-campuslearning experience, rather than being for a


page 25

separate group of students who learn at adistance. Therefore, the assurance of learningthrough distance and distributed methods andthe courses that adopt those methods is fullyintegrated within the University's more generalquality assurance processes.

106 For new programmes which are largely orcompletely delivered by FDL, the University hasput in place agreed mechanisms for theapproval of new teaching and learningmaterials, before they are put into operation.Staff with experience of FDL must be includedin the approval panel. The OCSLD providessystematic support for staff, through its series ofe-learning workshops, in order to address thenew requirements for the engagement with e-learning, as the University extends its use. Theaudit team considered that the University wasdeveloping appropriate procedures to manageand support these significant newdevelopments in the mode of learning.

Learning support resources

107 The SED explained the division ofresponsibilities between the Directorate ofLearning Resources providing central learningsupport resources, and the schools providingspecialist facilities such as laboratories, dedicatedresource rooms and technical staff. Provisionoperates in a range of modes and, in cooperationwith Student Services, is being enhanced to helpstudents who require extra learning support. Asthe result of criticism in internal and externalreviews, the University has recognised the needfor 'an enhanced programme of refurbishment' ofits central teaching spaces.

108 The library operates on three sites and,according to the SED, tries to meet students'needs by responding to, for example: thestudent satisfaction survey; feedback to subjectlibrarians at school boards and committees;electronic questionnaires; a survey of users andlearning resource issues raised in the course ofinternal and external reviews. In liaison withacademic staff the subject librarians monitorpatterns of demand and the audit team learntthat substantial extra resources were madeavailable in anticipation of the demand for

multiple copies of textbooks following theredesign of programmes for semesterisation.Subject librarians are also responsible for thedelivery of user education and the number ofstudents attending library support sessions ismonitored. There has been a recent expansionin the amount of information and the numberof electronic resources available to students onthe library's web pages. The team learnt that, inresponse to points raised in the SWS regardingsome aspects of the library service, staff wouldwork to improve provision of key resourcesthrough e-learning, digitisation, e-books and e-journals and academic staff would be askedto reconsider the format of reading lists inorder to encourage students to read beyondcore texts. It was pointed out in the SED thatstudents preparing dissertations andpostgraduate research students have access tothe Bodleian Library and other libraries withinthe University of Oxford. In association with theStudent Disability Service the library respondsto special needs by a combination of generaland individual provision.

109 It was noted in the report of the 2001audit that subject reviews had commented thatcomputer services were a particular strength,'with IT [information technology] provision ofhigh quality for students'. The audit team learntof a plan to increase investment in IT, mainly insoftware packages in liaison with the schools.According to the SED, Computer Services uses avariety of methods to obtain feedback onexisting and potential services and to identifystudents' needs, including an anonymous, on-line questionnaire. Academic computing officersand training officers in each school providesupport for teaching and the Media Workshopoffers pedagogical and technical support in theuse of the virtual learning environment (VLE).Although the SWS reported considerableenthusiasm among students for the VLE, it alsonoted some dissatisfaction with ComputingServices, in particular among postgraduatestudents. The team heard that, in response tothese criticisms, Computer Services is to increasethe number of central colour printers andplotters and to consider, in conjunction with theCentre for e-Learning, new forms of access to IT


page 26

services, including wireless connectivity, groupstudy facilities and booths for disabled students.Additional work stations for postgraduateresearch students will be provided in a newresearch building scheduled to open in 2006and Computing Services will investigatecomplaints by postgraduate students about thelack of subject-based IT facilities.

110 While students met by the audit team inthe course of the DATs confirmed that they werewell provided with learning support resources,the team also welcomed the constructiveresponse of the University to the criticismsexpressed in the SWS. The team endorsed theconclusion of the previous audit report that theLearning Resources Directorate 'provided a well-integrated service with a clear strategic aim ofidentifying and responding to students' needs'.

Academic guidance, support andsupervision

111 The SED explained that the Universityplaces great importance on the academic andpersonal support mechanisms for students andrecognises the challenges of providing supportfor a more diverse student population and ofpersuading academic staff of the importance ofstudent personal and academic support. TheUniversity was commended in the report of the2001 audit for its centralised approach tostudent support and its provision of guidancefor staff in assisting them to perform theirstudent support functions.

112 In the course of the DATs the audit teamlearned that induction arrangements which hadbeen commended in the previous audit reportcontinue to be thorough and now centreincreasingly on the engagement by studentswith their pages on the PIP. Students areprovided with all relevant information fromenrolment to graduation and through their PIPpages are encouraged to use their PIP pages 'asthe central mechanism for managing theirlearning'. The PIP enables students to view,amend and manage their personal andprogramme related information and to see theirtimetable, examination and assessmentschedules. It provides a virtual link with personal

tutors and automatic messages are generated ifstudents attempt to enrol on modules that arenot part of their programme. Links are alsoprovided to library resources and supportservices. Staff and students met by the teamspoke enthusiastically about their use of PIP andthe way in which it is being developed.

113 The SED explained that the academicsupport of students is the responsibility of arange of staff, including senior tutors in stage 1of the UMP, programme leaders or field chairs instage 2, personal tutors and members of theadministrative staff. All staff have access toSupporting Students: a Staff Handbook whichwas commended in the continuation auditreport. The Handbook, maintained by StudentServices, sets out roles and responsibilities forsupporting students and provides an invaluableresource to staff involved in addressing theproblems brought to them by students bydrawing together the strands of universityprovision in an accessible form. The on-lineversion, available from the beginning of the2004-05 session, contains links to other sourcesof information and guidance embedded in thetext. The Handbook emphasises the importanceof giving students accurate and up-to-dateinformation about semesterisation and asks staffto provide feedback to Student Services on 'thepleasures and perils' of using the electronicversion. It was the view of the audit team thatthe Handbook constitutes a valuable resource toacademic staff in guiding and support students.

114 The SED explained that all students areallocated to a personal tutor who is a memberof the teaching staff on undergraduateprogrammes or the programme leader forpostgraduate programme. The previous auditreport had found variability between schoolsand tutors in the effectiveness of the personaltutor system; the SWS also reported variabilityin the personal tutoring system, which studentsconsidered should function as 'the backbone ofthe support network within Brookes'. Followingthe previous audit report the University hadreviewed the work of personal tutors and, afterextensive consultation, a revised role forpersonal tutors has been approved which is


page 27

linked to the introduction of personaldevelopment planning in 2005-06. It isintended that personal tutors will provideessential support for PDP by encouragingreflection by students on the opportunities forpersonal development which are availablewithin their programme of study andengagement with careers advice and guidance.An on-line Personal Development Record andPlanner, accessible from the PIP, will supportthe new role of the personal tutor. From what itheard in meetings and read in thedocumentation supplied to it, the audit teamwas able to confirm the belief of the Universitythat the new policy linking personal tutoringand PDP will enhance academic and personalsupport for students.

115 According to the SED, Student Servicesmake available a range of written and web-based academic advice, covering such issues ascredit transfer, plagiarism and examinationtechnique, but it is admitted that the take-up ofgeneric study skills sessions - aimed primarily atmature students - has been patchy. AnAcademic Skills Centre began operating in2005 as a first point of contact for studentsseeking support for developing their academicskills; it will build on existing good practice inlearner support and will develop close links withschools in order to identify groups of studentsencountering difficulties.

116 The University was advised in the previousaudit report to continue to reflect carefully onthe implications of expanding postgraduateprovision. A themed audit of PostgraduateResearch Programmes conducted in 2002 maderecommendations to the University, the RDCand schools for the improvement of support forpostgraduate research students; in the course ofthe audit visit the audit team learnt that mostof the recommendations had beenimplemented. The creation of a GraduateSchool, a development confirmed during theaudit, will provide a focus for furtherimprovements. The responsibilities ofpostgraduate research students, supervisors andschools are set out in a Code of Practice whichis being revised to be fully compliant with the

Code of practice, published by QAA. TheGraduate Office provides university-wideinduction for new research students, supportedby high quality induction packs; students metby the team spoke warmly of the supportprovided by the Graduate Office. A teamconsisting of a director of studies and at leastone other supervisor provide supervision andsupport for research students; schoolpostgraduate research tutors are a furthersource of advice and support. Members of staffare not normally appointed as directors ofstudies before they have been involved insupervising two research students tocompletion and have attended a supervisortraining programme. Students met by the teampraised their supervisory teams highly. The RDChas the responsibility for ensuring that schoolsmonitor the progress of students on an annualbasis. A Research Students' Committee on whicheach school is represented meets regularly andreports to the RDC; a Postgraduate Society hasrecently been established by the SU to organisesocial events for research students. Although theSWS reported some dissatisfaction amongpostgraduates with learning resources, studentsmet by the team were appreciative of the qualityof library and IT services available to them;international students were appreciative of theavailability of free English language support.

117 Research students are required toundertake skills training appropriate to theirindividual needs and central provision iscombined with subject-specific trainingprovided by schools. A Brookes planner anddiary enables students and their supervisors toplan and record a programme of study to equipthem with research skills and also enhance theiremployment prospects after graduation. TheResearch Training Coordinator runs a series ofresearch student training seminars coveringtopics such as thesis writing, funding sourcesand applications, career planning and jobsearch skills. Attendance at training seminars ismonitored by the Graduate Office.

118 Research students who undertake teachingor demonstrating duties must be provided withappropriate instruction by their schools and


page 28

OCSLD provides training for those who teachfor more than 50 hours a year. Students whomet the audit team appreciated the trainingoffered and felt that the opportunity to beinvolved in undergraduate teaching enhancedthe value of their period of research studies.The team considered that the quality andintegrated nature of the supervision andsupport offered to postgraduate researchstudents constitutes an aspect of good practice.

Personal support and guidance

119 The SED explained that close links existbetween academic and personal support systems.The central student support services cover areassuch as counselling, health and dentistry, dyslexiasupport and study skills. A Financial Aid Officeadministers a new bursaries scheme to supportwidening participation and a Financial AidCommittee provides a useful forum 'foraddressing the increasingly complex and sensitiveissues of student funding'. Students can accesscentral support services directly or through theirpersonal tutors; the staff handbook, SupportingStudents, explains how to make referrals andcontains useful links to central services.

120 The SU offers a range of advice and supportfacilities for students and most of the University'scentral support service departments are locatedclose to the SU Advice Centre, 'to help access andto promote the close links between the twoservices to best meet student needs'. It isadmitted in the SED that there is scope for 'thereconfiguration of centrally provided information,advice and guidance for students'. In spite ofphysical restraints there is said to be a seniormanagement commitment to develop a 'onestop shop' approach to the delivery of studentsupport services but, in the meantime, there hasbeen an expansion of the range of electronicinformation and advice available.

121 An external audit of the University'sprovision of services for disabled students wascommissioned in 2003 and indicated that,although the Student Disability Service was wellregarded by students (a view confirmed in theSWS), more work was required at the level ofstructures and procedures to ensure that the

needs of disabled students are identified andresponded to. According to the SED, theStudent Disability Service is working with anetwork of newly appointed Equal Opportunityand Diversity Coordinators in the schools andwith OCSLD to increase awareness andunderstanding of the needs of disabledstudents and how to respond to them.Proposals for new programmes are scrutinisedin terms of availability of, and access to, anyspecialist facilities and groups are workingunder the direction of LTC to produce web andprint materials for teaching and assessing anincreasingly diverse student population and toplan and deliver appropriate staff development.

122 An International Student Advisory Service,whose work was commended in the previousaudit report, has had to respond to the needsof a rapidly increasing number of internationalstudents. The focus of its work is said to haveshifted from pastoral care to providing'accurate, reliable and timely information' toenquirers. The SED explained that it is intendedto monitor the reactions of internationalstudents to this move from face-to-face work toa greater use of telephone help-lines and web-based services, and also to monitor theirperception of how effectively they are supported.

123 The SED explained that, in order tomaintain the good employment record of itsgraduates, the University continues to give ahigh priority to employability in course designand delivery and in other supporting activities.An Employability Advisory Group reporting toLTC has been set up to oversee theimplementation of a recently approved CareersEducation Information and Guidance policy.Greater prominence is to be given to supportingstudents in the management of their study paththrough the curriculum and the link betweenpersonal tutors, and PDP is part of this approach.The Personal Development Record to supportPDP will have a number of links to material onthe web pages of the Careers Centre to supportwriting curricula vitae and preparing forinterviews, and a careers management skillsmodule will be available in 2005-06. A range ofcareers related activities in schools complements


page 29

central services, and several programmes in theUMP have independent study and placementmodules which offer opportunities related tofuture careers.

124 From its meeting with staff and studentsand from the documentation made available to it, the audit team was able to confirm thatthe University takes its responsibility for student support seriously and provides wellintegrated academic and personal supportsystems which are kept under review in orderto enhance their effectiveness.

Section 3: The auditinvestigations: discipline audit trails

Discipline audit trails

125 In each of the selected DATs, appropriatemembers of the audit team met staff andstudents to discuss the programmes, reviewed asample of assessed student work, saw examplesof learning resource materials, and studiedannual module and programme reports andperiodic school reviews relating to theprogrammes. Their findings in respect of theacademic standards of awards are as follows.

Business and management 126 The DAT covered two sets of programmes:four undergraduate, and one master'sprogramme. The four undergraduateprogrammes (BA (Hons) Business andManagement, BA (Hons) International BusinessManagement, BA (Hons) Business Logistics andBA (Hons) Business Innovation and Enterprise)form an integral part of the University-wideUMP, and are subject to its regulations. TheMSc Business and Enterprise conforms to theUniversity Postgraduate Framework and issubject to the regulations for taughtpostgraduate programmes. All theseprogrammes are located in the Business School.

127 The undergraduate programmes contain acommon first year and students choose theirsubsequent 'pathway' during the second

semester of study. There is an opportunity forstudent placement during year three of study inthe undergraduate programmes. The master'slevel programmes have a common core ofmodules which students met by the audit teamfelt gave them a valuable grounding to progress.

128 The DSED contained programmespecifications for all of the programmes whichcovered the learning outcomes expected fromthe programmes. The master's programmespecification included a mapping of thelearning outcomes as they are achieved in eachof the modules of the MSc programme. Thespecifications included reference to the relevantsubject benchmark statements including thegeneralist career entry type for the maste'sprogramme. In the meeting with staff of theBusiness School it was confirmed that staff havefamiliarity with the Academic Infrastructure.The programme aims and the learningoutcomes at both undergraduate and master'slevels are in line with the FHEQ.

129 The Business School uses a full statementon the procedures for preparing annual reviewswhich conforms to the guidelines in theUniversity's QSH. Using evidence from staff,students and external examiners' reports, thesereports are structured to report on thestandards of programmes and the quality oflearning opportunities and subsequently tocollate these issues into an action plan. Theyalso report on progress with the issues thatwere raised in previous years.

130 The annual review reports written forundergraduate and postgraduate courses providedata on applications, entry profiles, and cohortstatistics for the last 10 years. This data isprovided by the School Office. Summaries ofmodule results are included in an appendixtogether with details of graduate destinations. Inthe master's programme the data for the annualreview is collated by the programme leader usinglocal statistics. The audit team was told that thetrends in these statistics are raised in the relevantsections of the annual review reports and thatresponses would be expected if, for instance,high failure rates were reported in any module.


page 30

131 The School has also undertaken evaluationof the first-year experience in a survey whichhad responses from 160 students. This providedqualitative information on the best and worstfeatures of studying at Oxford Brookes. Agraduating students' exit questionnaire has alsobeen used in the School for three years andprovided comments including commendationof the intellectual level of the courses, the valueof the PIP system and the development of skillsthat students had achieved.

132 In line with university policy the Schooluses the module evaluation reports (MERs) tofeed back on the activity within individualmodules. These are based on various sources ofinformation including questionnaires tostudents, external examiners' reports, dataprovided from the central administrativesources and a critical commentary by eachmodule leader. The MERs are discussed at theBusiness School quality review meetings andthey contribute significantly to the preparationof the annual review reports.

133 Under powers delegated to it by QSC, theSchool has developed processes to complementthe University processes for designing andvalidating new programmes. An importantfocus in these programme approval proceduresis establishing the business case for a newprogramme proposal. The audit team notedthis additional feature as a valuable elementwhich might be more widely adopted in theUniversity procedures.

134 A periodic review was undertaken in June2004 for the taught postgraduate programmesin the Business School. This set out a number ofconditions, including recognition of the linkswith the benchmarks for master's degrees inbusiness and management, improvements tocareer guidance and clarification ofresponsibilities for collecting exit survey databetween the School and the University. Theundergraduate programme periodic review washeld in March 2005. The conditions from thiswere to provide matrices of learning outcomesand key skills against modules and to providean assessment schedule to be placed in the

programme handbooks. The School hasproduced an assessment configurationdocument which indicates the timing, type andweighting of assessments across all compulsorymodules. The audit team considered that theperiodic review process had been effective anduseful in supporting and enhancing the work ofthe School.

135 In the meeting with staff from theBusiness School the audit team heard that theprocesses for responding to external examinerswere being reviewed in the light of concernsabout inconsistency. At present externalexaminers receive a copy of the annual reviewreport but do not necessarily receive a specificresponse to any issue that they have raised. It isintended that in future external examiners willreceive a letter from the Assistant Deaninforming them of any actions that have beentaken and the timescale for the implementationof these actions.

136 In the Business School students can electto do a placement year and the students whowere met by the audit team were enthusiasticabout the advice that they receive when theychoose their placement and the managementof the process. The School has a dedicatedplacement office with two full-time staff. Priorto going on placement students take a moduleon skills profiling and placement search. Theteam were told that currently, depending whichprogramme they are on, students who chooseto take this module, and who are notsubsequently successful in finding a placement,may not be allowed to count this moduletowards their award. The School is aware of thisanomaly and is seeking a solution.

137 The audit team reviewed a range ofassessed work. The variety of assessment tasksand the appropriateness of the work was noted.The Business School has written guidelines forthe internal moderation of assessments whichillustrate the concern for good workingpractices. These include six differentapproaches which relate to the procedures forsituations when there are experienced assessorsincluded or not. There was evidence in most of


page 31

the written work that these guidelines werebeing followed. The School requires that allcoursework assignments and examinationpapers, once set, are checked by anotherappropriate member of staff. Marking criteriaare used effectively to enable students tounderstand how they are performing and in thework reviewed there was evidence of valuablefeedback to students about their work. Studentswere aware of the diagnostic purpose of someof their modules and valued the feedback onperformance that this provided. The standardof achievement in the programmes reviewed inthis DAT was appropriate to the titles of theawards and their location within the FHEQ.

138 An issue which has been raised by one ofthe external examiners on the undergraduateprogramme is the variation in the penalties forlateness that are applied to assessed work. TheSchool uses the guidance provided by theDirectorate of Academic and Student Affairs inthe 'Student Guide 2004-5' adopting one of threealternative models for penalising late work. Thechoice is determined by the module leader and isrelated to the type of work being assessed. Thesedecisions are agreed at the School Board. The useof different policies on late delivery of courseworkwas acknowledged by students who understoodthe reasoning behind it and were not aware thatit presented any problems.

139 Prior to enrolment, students met by theaudit team had received good qualityinformation and the master's-level students hadreceived a DVD with relevant programmedetails and pre-enrolment reading information.Students receive comprehensive programmedocumentation at induction. They appreciatethe complete coverage that this provides anddescribed it as accurate and full. They also haveaccess to the same information through theshared network drive used by the BusinessSchool. This provides a valuable and well-usedalternative to the printed material.

140 The audit team saw examples of thestudent handbooks for undergraduate andpostgraduate programmes and found these tobe clear and helpful. They provide full details ofthe structure of the programmes and the

assessment procedures. They also includeddetails of the subject benchmark statements.Students met by the team expressedsatisfaction at the level of information that theyreceive, describing it as full and useful.

141 Students have access to the PIP andexpressed great enthusiasm for the access andconvenience that this provides for investigationof module choices, assessment results and otherinformation. The students were complimentaryabout the resources that they used, includingthe excellent electronic access to journalarticles. They are readily provided with advicefrom subject librarians. The poor availability ofmultiple copies of text books was the only areawhich concerned the students that the auditteam met. Computing facilities were consideredto be adequate and the postgraduate studentshave access to dedicated computing space.

142 The audit team was informed thatsemesterisation had presented problems forsome students. The students felt that there hadbeen extra pressure on the time available fortheir studies during semester one caused by thelate change in the timing of the examinations.Second and third-year students regretted theloss of the reading week which had been auseful opportunity to revise for theirexaminations. In the meeting with staff theview was expressed that the Business Schoolwas keen to find a solution to the additionalpressure on students in the first semester andwould implement its own solution if a commonapproach across the University was not found.

143 Students are invited to express theiropinions on all modules using the MEQs. Thestudents met during the DAT were confidentthat their views were taken into account in therunning of the modules although they did notalways see immediate changes. The collations ofthe views expressed in the MERs are madeavailable to students' representatives throughthe annual School Quality Review meetings orthrough the annual review reports. (seeparagraphs 52-53 above). In the master'sprogramme the questionnaires have beendelivered electronically using WebCT. TheBusiness School has an undergraduate student

representative forum, although at the time ofthe audit visit the last minuted meetings werefor May 2004. Student representatives areappointed from programmes and sit onprogramme and School committees. Therepresentatives receive training and guidancefrom the SU although in the meeting with theaudit team the representatives expressed thewish that there could be an improvement in themechanisms for obtaining feedback commentsfrom the students that they represent.

144 Students felt well supported by the staffwho were accessible and approachable. Thelearning resources were well used and effectiveand despite initial problems with semesterisationthe students felt confident in their ability toachieve and progress in their programmes. Thestudents also expressed the view that they feltwell prepared for their future careers. The auditteam concluded that the quality of learningopportunities offered in the business andmanagement area was suitable for theprogrammes of study leading to the awards.

English studies 145 The DAT was based on the followingprogrammes: BA Single Honours, English Studies;BA Combined Honours, English Studies;Postgraduate Certificate/PostgraduateDiploma/MA Modern and Contemporary Poetry.

146 The DAT was supported by a dedicatedDSED prepared for the audit, together withprogramme specifications. The DSED statedthat all three programmes had been designedwith the requirements of the Subject benchmarkstatement for English and the FHEQ taken intoaccount. The DSED also stated that theprogrammes were designed with reference toUniversity strategic priorities. The audit teamidentified appropriate references to theseelements of the Academic Infrastructure in theprogramme specifications and in the variousmodular handbooks. The DSED made referenceto the School of Arts and Humanities' QualityManual and the School's Academic Quality andStandards Committee which ensures thatSchool and Department practice are in line withUniversity policy. Through discussion with staff

and by reviewing the available documentation,the team was able to establish that thesereferences were appropriate and wereunderpinned by firm knowledge andunderstanding of the content and purpose ofthese reference points.

147 The DSED did not include progression andcompletion data for the three programmes.Achievement in terms of degree classification ofgraduating students was given in the DSED butnot analysed. Some commentary on entryqualifications was provided. The DSED didconfirm that the appropriate data is availablecentrally. The audit team explored with staffhow these statistical data were used. Overall,however, the team had some difficulty inestablishing a transparent connection betweenstatistical data and the monitoring of qualityand standards. The DSED made the point thatthe Department recognises that the studentfailure and withdrawal rates at the end of thefirst year of the undergraduate programme arehigher than they would wish. The DSEDexplained that this problem was considered inthe redesign of the programmes and in thesubsequent Periodic Review and that thedecision was taken to revise the Stage 1provision in order to respond more effectivelyto the particular needs of the students. Indiscussion with staff and with scrutiny of thenew modules, the team was able to confirmthat the new stage 1 provision had beendesigned in careful consultation with externalexaminers and with the English Studies SubjectCentre and that the new curriculum does haveappropriate content, learning outcomes andassessment for the programmes. TheDepartment has plans to evaluate theeffectiveness of the provision at the end of thefirst year of its implementation.

148 The School and Department have wellestablished quality procedures for programmeevaluation and review. These include annual andperiodic review of programmes and studentevaluations on module delivery. APR andperiodic review reports are considered by theschool QSC. Evidence from student feedback onmodules is incorporated in these reviews. The


page 33

APR is the forum for the programme to reflecton standards and the quality of the provision.The outcome from the APR is an action plan forenhancement of the provision and theremedying of any identified issues. The auditteam, through discussion with staff and students,was confident that this system was effective.

149 The audit team was able to studyprogramme external examiners' reports for thelast three academic years. Numerous positivecomments were noted, both about theacademic standards achieved by the studentsand about the quality of the teaching andlearning environment through which staff helpstudents to reach their potential. Where mattersfor consideration had been raised by externalexaminers, the team found evidence of positiveresponse in accordance with the expectationsof the relevant University policy. The teamnoted that there had been a problem with thetiming of the examinations process in thepreceding semester which had resulted ininsufficient time for the external examiner forthe Master's Programme in Modern andContemporary Poetry properly to scrutinisestudent work. The team saw and heardevidence to show that the Department wasfully aware of the problem and was takingmeasures to remedy it.

150 The DSED stated that assessmentprocedures comply with the benchmarkstatement that students should experience avariety of forms of assessment. The audit teamwas able to confirm that this was the case andthat the assessment procedures were appropriateto achieving the learning outcomes specified.

151 Students give feedback on the assessment,content and delivery of modules through themodule evaluations. These are followed up inthe programme annual report and may becomepart of the action plan if changes to the systemare deemed as necessary. The Department hasrecently introduced a standard pro formacoversheet for staff to provide feedback tostudents on assessed work. This was in responseto the view expressed by students in theperiodic review that written feedback onassignments was inconsistent across tutors.

152 The audit team reviewed a substantialvolume of assessed student work from a rangeof modules within the three programmes. Theteam noted that the standard of achievementwas underpinned by a range of appropriateassessment and assessment related processes.These included clarification of learningoutcomes and assessment requirements inmodule handbooks, appropriate double markingand monitoring arrangements and appropriateinteraction with external examiners in designingsuitable questions. The team reached theconclusion that the nature of the assessmentand standard of student achievement in theprogrammes were appropriate to the title of theawards and their location within the FHEQ.

153 All undergraduate students receive a copyof the English Studies Student Handbook at thebeginning of their course and a modulehandbook is provided for each module withstandardised contents. MA students receive theStudent Handbook: Humanities TaughtPostgraduate Programme in advance of theirenrolment. Postgraduate modules also havetheir own handbooks. The DSED stated thatcommunication with students is done largelythrough email and the PIP. Students were verypositive about the Student Handbook, modulehandbooks and the PIPs provision andconfirmed that communication systems wereeffective in the Department. The one area thatwas mentioned for improvement was that noinitial reading lists were sent out to studentsprior to the commencement of their degrees.

154 The DSED stated that student evaluation isa very important element of quality assuranceand enhancement but it presented littleevidence of how the system of studentrepresentation works within the programmes.Student representatives are invited to attendDepartment meetings to represent studentviews but attendance varies. There is nomention in the DSED of how the Department'ssystem of student representation mirrorsUniversity systems but the audit team was ableto confirm through discussion with staff andstudents and through appropriatedocumentation that the system is in line withUniversity policy and is working effectively.

University of Ulster

page 34

155 The DSED acknowledged some problemswith the physical resources in the programmes.Students have expressed concerns in the APRregarding the availability of secondary texts andin the resources of the Headington Librarywhich serves the programmes. The Departmentis continually reviewing the situation and hasimplemented a temporary loan system for coretexts. In addition, level 3 undergraduates andpostgraduates do have access to other localresources such as the Bodleian Library.

156 The audit team noted the commitment andengagement of the subject staff, and by theatmosphere of mutual respect within which staffand students, both undergraduate andpostgraduate, collaborate. The team concludedthat the quality of learning opportunities availableto students was suitable for programmes of studyleading to the respective awards.

Construction management and urban design 157 The School of the Built Environmentcovers five discipline areas: architecture; civilengineering; planning; real estate andconstruction. The DAT covered the twoundergraduate programmes BSc (Hons)Building and BSc (Hons) ConstructionManagement both of which are located withinthe Department of Real Estate and Constructionand are also contained within the UMP. TheDAT also included the MA in Urban Designprogramme which is located in a joint centrebetween Architecture and Planning. Thedocumentation provided for the DAT wasspecifically developed for the audit, andprepared by senior staff supported by courseteams and with comments from students.

158 Programme specifications describe thecourse and provide information on the courseaims, learning and assessment strategies andshow how the programmes link to theUniversity's strategies of linking research intoteaching, providing an international contextand encouraging reflective learning. Thespecifications also provide reference to theappropriate benchmark statements which arethen expanded in the key areas of knowledgeand understanding, professional skills, and

transferable skills. In the postgraduate coursethese are mapped across modules. Furtherdetailed information is provided on individualmodules in the comprehensive coursehandbooks. Students considered the inclusionof the industrial placement a particular strengthof the Construction Management course whichgreatly increased their employability.

159 Progression and completion data areprovided by the University's Systems Office andmade available at the time of a programme andschool review. In the DSED some informationwas provided on completion rates for the MAUrban Design course but there was noevaluative commentary. Following anexamination of the school review documentsthe audit team considered that the reviewprocess would be enhanced by a more detailedanalysis of the statistical information provided.

160 The process for monitoring and reviewingcourses is through a series of programmeannual reviews which in turn inform an annualreview of the School. In addition there is aperiodic review of programmes at departmentlevel. The evidence available to the audit teamshowed that the periodic review had involvedexternal representatives, and had generallybeen an effective process. The review hadincluded discussion of the issue of group work(see below). While the Department hadresponded to the matters raised by students inthe course of the review, it was not clear to theteam that the concerns raised by students hadbeen fully resolved.

161 External examiners provide an importantexternal view both of the course and of theacademic standards achieved by students.External examiner reports are received in theSchool by the Assistant Dean who provides aninitial response acknowledging receipt of thereport and the issues that need to be addressed.A full response is considered at the programmeannual review. External examiners are asked toconfirm each year that previous issues have beenaddressed. Recent external examiners haveexpressed satisfaction with the programmes.


page 35

162 For the undergraduate programmes awide range of assessment methods is employedincluding essays, reports, portfolios, reflectivediaries, on-line quizzes as well as moretraditional unseen examinations. Assessmentcriteria are provided to students at a generallevel through the module handbooks, withadditional information provided whenassignments are issued and supplemented bydiscussion by tutors. Students were generallysatisfied that they knew what was expected butnoted that feedback on assessed work couldtake much longer than the two weeks normallyexpected. Students expressed some concernover the assessment of group work which staffexplained would contribute a substantialamount of about one third of theundergraduate programme and a little less onthe postgraduate programme. Typically, groupwork might account for 40 per cent of amodule but students considered that individualwork carried out thereafter could be affected bya poor group performance. The studentperception was that individual contributions togroup work were not always recognised andthis might impact on their final grades. Staffmet by the team confirmed that there were fewexamples of a student failing assessed groupwork, as good students compensated for thepoor achievement of others. The School isaware of the concerns of students and attemptshave been made to address these concerns.During the audit the team saw a number ofdifferent approaches to assessing group workincluding examples of good practice in otherDATs. The team recognises that the assessmentof the individual contribution to group workhas the potential to adversely affect therobustness, transparency and fairness of theassessment of individual student work and theschool is therefore encouraged to continue toreview its practices with regard to assessmentof group work. Having considered the range ofassessed work provided, the team found thatthe standards of student achievement wereappropriate to the titles of the awards and theirlocation with the national FHEQ.

163 The student handbook is a comprehensivedocument that provides a wealth of usefulinformation to students. In addition to detaileddescriptions of each module there isinformation on course structures, summaries ofregulations, advice on study skills, informationon student support, learning resources etc. Theprinted materials are supported by PIP whichwas considered to be both reliable and useful.Students were satisfied with the resourcesavailable to them within the School and University.

164 Until recently the collection of studentfeedback had been recommended by theUniversity but had not always been undertakento avoid evaluation overload. The currentposition is that feedback is to be collected forevery module. The feedback was generallysupportive with concerns normally aboutroutine matters over which there is little controlsuch as timetabling issues. Apart from the issueconcerning group work, noted above, thestudents did not have serious concerns.

165 Programme and school studentrepresentatives attend and contribute to theannual review meetings. However, the Schoolrecognised 'that this process is not perfect anddiscussions are underway as to how to improvestudent participation'. Students also attend fieldcommittee meetings, and were satisfied thatthey have ample opportunity to voice theirviews and concerns.

166 The audit team found that the quality ofthe learning opportunities available to studentswas suitable for courses of study leading to thenamed awards.

Mechanical engineering 167 The DAT was based on the subject area ofmechanical engineering located in the School ofTechnology. The DAT covered two undergraduateprogrammes the BEng (Hons) AutomotiveEngineering and the BEng (Hons) MechanicalEngineering. It was supported by a comprehensiveDSED prepared for the purposes of the auditand supplemented by a range of appendices.


page 36

168 The DSED included programmespecifications that reflected an appropriateengagement with the FHEQ although they didnot refer explicitly to it. They did, however,make appropriate reference to professionalbody accreditations. Again, although no explicitreference was made to the relevant subjectbenchmark statements for this subject atundergraduate level, an appropriateengagement was reflected in the design andstructure of the programmes. The subject teamwere aware that a review of the programmeswill be necessary following the introduction ofthe UKSPEC by the Engineering Council andwere already engaging in the process followinga recent accreditation visit by the Institution ofMechanical Engineers.

169 Progression and completion data areanalysed as part of the accreditation exercise.Although, however, data were included in theannual reporting cycle, it was unclear whetherany critical analysis had been undertaken as thiswas not presented. Data on progression andcompletion was presented and discussed atExamination Boards and discussed by the courseteam. Quality assurance at a local level takes placein accordance with the School of Technology'spublished procedures, which are based on theinstitutional framework, and the audit teamfound that these were operating effectively.

170 The external examiner has an appropriatelevel of involvement in both the assessment andreview of programmes. External examiner reportsconfirm the overall standards of the award andthe reports are considered as part of the annualreview process, with a formal response from thecourse team to any issues raised.

171 Students are involved in qualitymanagement systems through representationon field committees and this is supplementedby close relationships between the staff andstudent body. The audit team was informed bystudents that the staff team were highlyaccessible and that the students appreciatedthis close contact. Student opinion on thequality of teaching is also sought throughmodule questionnaires and staff are required topresent to the subject examination committee

an account of the module evaluation. Thestudents indicated a high level of satisfactionwith their teaching, support and learningresources and confirmed the positiveexperience that staff teams offered to students.Actions on matters raised through liaisoncommittees or student questionnaires are dealtwith in an appropriate manner and appropriatefeedback to students is provided.

172 Semesterisation appeared to have causeda number of problems which had affected thequality of the teaching and learning experienceof students in the first year of operation. Theteaching pattern had been affected by theperceived reduction in delivery time from threeterms to two semesters and the assessmentshad become bunched causing extremeproblems around hand-in dates. Also, itappeared that in a number of cases the studenttimetabled contact has been compressed withinthe teaching week resulting in attendance overan extended day spanning the three scheduledslotting periods. Students felt that staff wereaware of the problems and had acted in anappropriate manner to resolve those issueswhich were under their immediate control.

173 The audit team reviewed a range ofassessed student work and was satisfied thatthe nature of the assessment and the standardof student achievement met the expectations ofprogramme specifications, and was appropriateto the relevant awards and their location withinthe FHEQ. However, the feedback on thestudent work provided varied widely in itsquality and usefulness. Students did confirmthis variability and they also felt that feedbackcould be more timely.

174 Students are provided with student andcourse handbooks. The course handbook iscomprehensive, and students informed theaudit team that they receive an appropriatelevel of information in order to facilitate theirstudies. The module descriptors contain detailsof the learning outcomes and assessmentmethods. However, it was unclear how theseare directly related to the curriculum and theprogramme team may wish to review this areaas part of any revision to the handbook.


page 37

175 In terms of learning resources the auditteam was informed that there were no issueswith either the provision or the quality ofmaterials available to the students. The teamalso heard about the use that was being madeof the PIP. Staff and students bothacknowledged the usefulness of the portal butalso felt that since their programme was highlyprescriptive in nature they did not utilise its fullfunctionality. However, the ability to accessresults was felt to be very useful. Students alsopraised the use of e-learning through WebCT asan aid to their learning.

176 The programmes also offer to students theoption of taking an industrial placement duringtheir third year of studies. The placements weresupported by a comprehensive programme ofseminars and a dedicated tutor. Studentspraised the support that they had received bothbefore and during their placements and theaudit team considered that this was an exampleof good practice.

177 Externality is a key feature of theprogrammes within this area. The programmesoffered are highly vocational and benefit fromaccreditation from an appropriate professionalinstitution. They also benefit from close contactwith industry through an industrial advisorypanel. However, the meetings of this board hadrecently lapsed and the team were keen torevitalise these links. Clearly both industrial andprofessional inputs contribute to the maintenanceof quality and standards within the programmesand this is reflected in the high levels ofemployability that the subject area attains.

178 The audit team was satisfied that thequality of learning opportunities provided forstudents was suitable for programmes ofstudies leading to the named awards.

Philosophy and religious studies179 The Philosophy and Religious Studiescombined honours programmes are offeredwithin the UMP and delivered by the academicgroup of Philosophy, Theology and Religionwithin the WIE, a school formed after themerger of Westminster College with theUniversity in 2000. As this subject area has yet

to undergo periodic review, a DSED wasprepared for the DAT following extensiveconsultation with staff and students. Thereligious studies programme was validated in2000, was approved in a QAA subject review in2001, and was redesigned in preparation forsemesterisation in 2002; the philosophyprogramme was validated in 2001 andredesigned in 2003. According to staff seen bythe audit team the transition to semesters hadbeen unproblematic and students, whowelcomed the increased flexibility involved inthe change, confirmed this view.

180 The programme specifications attached tothe DSED make full reference to the relevantbenchmark statements and link learningoutcomes to the FHEQ. They are aligned with theUniversity's strategic priorities of linking researchand teaching, internationalising the curriculumand encouraging reflective independent learning.

181 Data on progression, completion and firstdestinations of graduates were included in theDSED but it was explained to the audit teamthat detailed analysis for strategic purposes takesplace at school level. Programmes are reviewedannually: programme committees considerMEQs and reports from external examiners anddraw up an action plan for the followingsession. In the current session local programmedirectorate audits have been introduced by theInstitute to supplement programme review, toidentify good practice and encourage reflectionon strategy in order to agree a developmentalfocus for the following session.

182 The audit team heard from staff that theyenjoyed and valued highly close contact withtheir external examiners. External examinersreport orally to Subject ExaminationCommittees and Module Examination Boardsand their written reports are considered in thecourse of annual programme reviews anddirectorate audits. They are sent copies ofannual reviews which contain a summary oftheir comments and the response of theprogramme team. Recent advice from externalexaminers has resulted in the introduction of aplagiarism disclaimer on submitted work andthe videoing of assessed presentations. Reports


page 38

seen by the audit team confirmed theconstructive and complimentary nature ofcomments from external examiners.

183 Both programmes have produced a LTASaligned with the LTAS of the Institute. A rangeof assessment strategies is employed to achievethe learning outcomes laid out in theprogramme specifications. Students met by theaudit team confirmed that they were familiarwith learning outcomes and assessment criteriafrom programme handbooks and moduleguides. They appreciated the prompt, detailedand helpful feedback provided on assessments.The team reviewed a range of student workfrom stages 1 and 2 of the programmes andfound that the Institute's procedures for theassessment of students' work had beenfollowed and that the standard of studentachievement was appropriate to the titles of theawards and their location within the FHEQ.

184 Students are provided with programmehandbooks containing programmespecifications, module descriptions, assessmentcriteria and both programme-specific andrelevant University policies. Guides for eachmodule include the details of the level andstatus of the module, its learning outcomes, anassessment schedule, guidance on referencingand recommended reading. Students met bythe audit team found the handbooks andmodule guides useful but valued their PIP pageseven more highly as a source of information.Staff confirmed that PIP has become anessential tool in communicating with students.

185 It was explained in the DSED that theteaching accommodation available forphilosophy and religious studies was appropriateand well equipped. Both programmes benefitfrom the advice and support of a subject librarianand specialist audiovisual and IT staff. Studentsmet by the audit team were enthusiastic aboutthe availability of books and computers and thehelp that they were given in using electroniclearning resources. The learning resource theyappreciated most highly, however, was theiraccess to research active members of staff whowere also innovative and responsive teachers.Staff met by the team confirmed that WIE has a

comprehensive plan for staff development andrequirements are identified in the course ofannual Personal Development Reviews. AnInstitute scheme for the PELT had beenintroduced at the start of the session and staffconfirmed the claim in the DSED that PELT is'broader and richer' than traditional schemes forthe peer observation of teaching. Staff worktogether in pairs to enhance teaching techniques,such as the assessment of oral presentations orthe conduct of seminar discussions. The teamconsidered this initiative to constitute an exampleof enhancement which the University might wishto consider adopting more widely in support ofits learning and teaching strategy.

186 All students are assigned a personal tutor toprovide academic and personal support andguidance. Among the students met by the auditteam those who were mature and part-time saidthat they had received specialised help andadvice from their personal tutors. Staff told theteam of the preparations made in the Institutefor the introduction of PDPs and the intendedlinks with personal tutors. In addition to theirpersonal tutors students are offered academicguidance by the senior tutor in stage 1 and byfield chairs in stage 2. Students found theacademic staff very accessible and experiencedno difficulty in finding help when they needed it.

187 Staff seek feedback from students by meansof MEQs; those seen by the audit team werewell-designed and gave the opportunity forevaluative comment. In addition the Institute hasintroduced a version of the student courseexperience questionnaire which generatesfeedback on programmes as a whole. Feedbackis also sought from student representatives onprogramme committees and other Institutecommittees, although students met by the teamfound the representative system rathercumbersome and they preferred to raise mattersdirectly and informally with staff. The team wasprovided with a recent example of the activeinvolvement of students in quality management.An Institute-wide compulsory module, HumanDevelopment and Learning, introduced as partof the redesign of programmes forsemesterisation has been modified considerably


page 39

as the result of student feedback which indicatedthat it had placed too great an emphasis onstudy skills and did not provide enoughopportunity for discussion of academic issues.

188 The audit team was satisfied that thequality of learning opportunities provided forstudents was suitable for combined honoursprogrammes in philosophy and religious studies.

Section 4: The auditinvestigations: publishedinformation

The students' experience of publishedinformation

189 The information available to the audit teamincluded the SWS, student handbooks,prospectuses and an extensive set of on-lineinformation including PIP. The team met withstudents during the briefing visit and with groupsof students during each of the DATs. The teamalso discussed some of the issues with staff bothin the institutional meetings and in the DATs.

190 The student experience as described in theSWS is the result of a comprehensive and wide-ranging survey followed by detailed analysisand review (see paragraph 19 above). Themajority of students considered that the pre-arrival information about the University wasuseful (96 per cent) accurate (83 per cent) andclearly written (96 per cent). However,information about the course was consideredless accurate and 30 per cent of students feltthat the information provided about theircourse was not accurate. The SWSacknowledged that this may be a result ofinaccurate recollection as students were beingasked to comment at least nine months afterreceiving the information and may be as mucha reflection of expectation as accuracy.

191 Students met by the audit team consideredthat in general the information provided centrallyby the University was accurate but that this wasnot always the case with schools, often becauseof the need to keep it up to date and accurate.The University states clearly in its advice to

students that current information is located onthe PIP pages, but it appeared that this messagedid not always get through to all students.During the period immediately prior to the auditthe University underwent substantial changes(see paragraphs 25 and 26 above), including adecrease in the elective choice of modules forsome students. The students noted that some ofthe changes to course structures had affectedtheir choice of programme. This was usuallyattributed by students to semesterisation thoughmore properly it was the decision to reduce thenumber of modules which happened at the sametime as restructuring.

192 Although the University made substantialefforts to ensure that students were properlyinformed about the changes by a variety ofmethods there were some students who werenot clear about aspects of their course. Thisappeared to be a greater problem for part-timestudents, who spend less time in the Universityand rely more on printed materials. Studentsgenerally have a good working relationshipwith staff and this had clearly helped studentsto resolve uncertainties, but this does requirethe students to know that there is informationthat they are missing or that the informationthat they have is out of date.

193 The PIP pages were generally consideredby students to work well and give access to awide range of information, allowing students toregister for modules, find out marks forassessed work, etc. When the on-line systemidentified problems with module choices itautomatically notified tutors so that promptaction could be taken. Students generallyconsidered PIP to be an excellent system.

194 The University is aware of the currentissues concerning the currency of informationand has taken action both at University leveland at school level to address them.

Reliability, accuracy and completenessof published information

195 The University has put in place thenecessary procedures to ensure that informationrelating to TQI is placed in the Higher Education


page 40

and Research Opportunities in the UK (HERO)website. External examiners are required tocomplete a revised report from which summariesare prepared and the majority of these reportsare now in place. Programme specifications forundergraduate programmes were revised as partof the semesterisation process. These arecurrently being collected in the University webpages before being linked to the HERO website.Similar arrangements are being made forpostgraduate programmes, where about 50 percent of the external examiner reports have beenpublished, and the remainder were expected tobe dealt with before the required deadline.Periodic reviews of both subject areas andcollaborative partnerships are being provided toHERO on an ongoing basis as they are prepared.Progress towards meeting the TQI requirementsis being monitored through the QSC.

196 Additional information provided to studentsincludes prospectuses and course handbooks.These are available both in printed form and on-line. A difficulty that the University recognises isthat printed information can become out of dateparticularly in a rapidly changing environment -as during semesterisation. The University hastaken steps to keep printed information as up-to-date as possible and maintains updatedinformation on the websites. All courses wererequired to have new course handbooks for2004-05 to ensure that they were accurate atthat time with further revisions being maintainedon the on-line versions available to studentsthrough their PIP pages. The Directorate ofCorporate Affairs oversees the centrally generatedinformation and monitors school activity for keydocuments such as course information. Schoolsthen take responsibility for ensuring thatinformation (both printed and on-line) is accurateand complete.

197 Overall, the audit team found that theUniversity had made positive progress towardsmeeting the requirements for publishedinformation, and was confident that theUniversity would be in a position to satisfy therequirements laid out in HEFCE 03/51,information on quality and standards in highereducation: Final guidance.


page 41

Findings

Findings198 An institutional audit of the OxfordBrookes University (the University) wasundertaken during the week 25 to 29 April2005. The purpose of the audit was to providepublic information on the quality of theUniversity's programmes of study and on thedischarge of its responsibility as a UK degree-awarding body. As part of the audit process,according to protocols agreed with the HigherEducation Funding Council for England(HEFCE), the Standing Conference of Principalsand Universities UK, five discipline audit trails(DATs) were conducted. This section of thereport of the audit summarises the findings ofthe audit. It concludes by identifying features ofgood practice that emerged during the audit,and making recommendations to the Universityfor action to enhance current practice.

The effectiveness of institutionalprocedures for assuring the quality of programmes

199 Since the time of the continuation auditcarried out by QAA in 2001 the University hasreviewed and streamlined its committeestructure. The Executive Board (EB) deals withmatters of strategy and planning and theAcademic Board (AB) is responsible for the qualityand standards of the academic programmes ofthe University. The AB works through a numberof committees, of which the Quality andStandards Committee (QSC) and the Learningand Teaching Committee (LTC) have primaryresponsibility for the taught programmes.

200 The overall structure of the managementof quality and standards is for the delegation ofthe primary responsibility to the schools,working within overarching principles set outby QSC and LTC and subject to monitoring andreview by QSC. The schools have their owncommittee structures and, for example, theirown rules of assessment. A central body, theAcademic Policy and Quality Unit (APQU),supports the QSC. It is led by the Head ofQuality Assurance (HQA) and some of its staffmembers act as Quality Assurance Officers

(QAOs) in the schools, so acting as linksbetween schools and between the eight schoolsand the central committees.

201 The AB delegates a significant degree ofresponsibility for the quality processes to schoolboards which, subject to the constraints set outin the Quality and Standards Handbook (QSH),can choose to operate processes in a mannerappropriate to the requirements of thediscipline. Schools report formally to QSC, butin the case of annual programme reviews QSC isonly presented with a summary by the APQU.Also, school board reports and those of QSCand AB's other subcommittees are circulatedelectronically with the intention that ABdiscussion can be focussed on specified agendaitems. The combination of these arrangementsdistances AB from quality assurance processes atthe point of application. The audit team judgedthis to be a weakness in AB's management ofthe quality of the University's programmes andadvise that reporting processes to AB bereviewed. The team would also encourage theUniversity, in its ongoing review of theoperation of its committee structures, to includeparticular attention to ways in which the AB isable to address and fulfil its terms of reference.

202 The principles guiding the management ofquality and standards are set out in a QSH. Thisis available on-line and contains detaileddescriptions of the procedures for the approvalof new or revised programmes, periodic andannual review, external examining, creditaccumulation and transfer and expectations forprofessional, statutory and regulatory body(PSRB) review and accreditation. The necessarypro formas for all these activities are included asappendices to the handbook.

203 The approval process for new programmeshas two stages, the first of which is theconsideration of the resource issues and thedetermination that the proposed course of studyfits the University's strategy. Following this, adetailed academic case is prepared by aProgramme Development Team which is requiredto take external advice during its considerations.The school then establishes an Approval Panel


page 44

which includes academic staff from outside theschool, a representative from APQU, and usuallya member external to the University. Theoutcome of the process is considered by theschool board, by the panel and by APQU. Ifagreed, the outcomes are then recommended toQSC after any conditions have been met. Majorchanges to existing programmes must followessentially the same procedure.

204 The University uses a periodic reviewprocess focused on subjects or disciplines andcarried out on a six-year cycle. The reviews aremanaged by the school but the panel whichconducts the review must include academicstaff from other schools, a representative fromAPQU, often including the HQA, and a memberexternal to the University. The panel report andthe school response are considered by the QSC,providing assurance at the institutional level.

205 Annual programme review is alsomanaged within the school. Each programmeteam compiles an annual report taking accountof student feedback and external examiners'comments and identifying any examples ofgood practice. These reports are drawntogether as part of the school annual review,conducted at a specific review event. Theschool annual review reports are collated by theAPQU and issues arising addressed to therelevant area of the university. While effective atschool level, this process does not make bestuse of the reports which merit widerdistribution and discussion.

206 The audit team noted also that theabsence of a mechanism for discussion of thesereports with academic staff from other schoolsweakens the effectiveness of the spread of goodpractice between schools. It is desirable thatspecific action be taken to use this additionalopportunity for the spread of good practicebetween schools.

207 The APQU also conducts a series of'themed audits' on behalf of QSC, concernedwith mechanisms for assuring quality across theparticular aspects of the University's work. Thereports are received both by the QSC and byappropriate school committees to enable them

to be acted upon. The audit team judged theseaudits and their reports to be an example ofgood practice.

208 Students have representation on all theUniversity's major boards and committees otherthan the EB. Direct feedback on modules formspart of annual programme review and isenhanced by student representation onprogramme teams and the school boards.Schools work closely with relevant PSRBs whichmay be involved with periodic reviews and,where possible, programme reapproval iscombined with accreditation.

209 The University has carefully reviewed andrevised during recent years its processes for themanagement of the quality of its programmes.The self-evealuation document (SED) presentsthese changes as nearly complete but notessome areas where consideration is ongoing, forexample, on information systems and thefinancial and strategic planning model.

The effectiveness of institutionalprocedures for securing the standardsof awards

210 The role of external examiners is 'providingthe principal…regular external view of qualityand standards of the academic programmesoffered by the University'. Examiners are expectedto monitor assessment processes within theinstitution and consider overall academicstandards. Schools are responsible for nominatingexternal examiners and nominations areapproved by the school board prior tosubmission to the APQU. QAOs makerecommendations to the QSC on the suitability ofnominations, which approves these on behalf ofAB. On appointment, APQU provides initialinduction to new external examiners through itsWebPages. External examiners are required toattend Subject Examination Boards and to submitan annual report, the template of which has beenextensively revised. Reports are received by APQUand disseminated to schools. The Dean isrequired to ensure the external examiner receivesa formal response and schools have adoptedvarious procedures to ensure external examinersare informed on the outcomes of their reports.

Institutional Audit Report: findings

page 45

211 External examiner reports are synopticallysummarised in an annual report by the QAOsand this overview is considered at Universitylevel by QSC on behalf of Academic Board. Theaudit team reviewed a number of exampleswhich demonstrated the responsive nature ofannual reports to external examiners' reports,and linked clearly to the institution'senhancement agenda.

212 The audit team noted that statistical datawas appended to annual review reports,however, there appeared to be no requirementto analyse and comment on this and no criticalappraisal was provided. Although statistical datawas considered at programme level and specificthemes appeared to have been analysed at boththe QSC and LTC, the University does notappear to have adopted an overall strategy forthe analysis of data at an institutional level. Italso stated that, although periodic review panelsare provided with statistical evidence, not allpanels consider this information in detail.

213 The relationship with external examinersprovides not only assurance of standards butalso a frequently used resource forenhancement through curriculumdevelopment. The audit team considered thatthe policy of encouraging systematic anddetailed external examiner reports followed bysynoptic internal summaries recording evidenceof consequent actions functions well.

214 Overall, the audit team found that theUniversity had effective mechanisms in place toassure the standards of the awards offered. Theteam would encourage the University to adoptrigorous procedures to ensure the analysis ofdata at all levels as an important element in themaintenance and enhancement of standards.

The effectiveness of institutionalprocedures for supporting learning

215 The responsibility for the provision oflearning support resources is divided with theDirectorate of Learning Resources providingcentral services and the schools providingspecialist, subject-specific support. Library andinformation technology provision is operated

across three sites, is monitored regularly and isresponsive to the needs of an increasing diversestudent body. There are plans to make furtherinvestment in the provision of electroniclearning resources and to extend the use of thevirtual learning environment. In response tocriticism in internal and external reviews, theUniversity is refurbishing and upgrading centralteaching accommodation.

216 The University recognises the challenge ofproviding academic guidance, support andsupervision for a diverse student population andof persuading academic staff of the importanceof personal and academic support for students.Increasingly pages on the Personal InformationPortal (PIP) are becoming the centralmechanism for managing student learning fromenrolment to graduation. Students are providedwith a variety of information and links tosupport services on their PIP pages which servealso as a 'virtual office door' for their tutors.Academic staff are expected to be accessible tostudents and are supported in their academicand pastoral roles by an on-line handbook,Supporting Students, which draws together thestrands of provision in an accessible form toenable staff to provide students with accurateinformation and informed guidance.

217 Postgraduate research students, inparticular, are well provided with guidance andsupport by the Graduate Office centrally and bysupervisory teams within their schools. Thissupport is about to be enhanced by theestablishment of a Graduate School and theprovision of work stations in a new researchbuilding. Research students who take onteaching responsibilities are provided with anappropriate combination of central training andinstruction in their schools.

218 Close links exist between academic andpersonal support systems. Student Services andthe Students' Union (SU) provide a range ofservices centrally. The Student Disability Serviceis working with schools to increase awarenessand understanding of the needs of disabledstudents and the International Student AdvisoryService is responding to the needs of rapidly


page 46

increasing numbers of international students. Inview of the importance attached by students totheir employability after graduation increasingprominence is being given to supportingstudents in the management of their careers.Personal tutors are to play a key role insupporting personal development planning andthere will also be close links to the CareersService. The general accessibility and helpfulnessof staff in academic and administrative matterswas strongly appreciated by students.

219 Two key objectives of the University are todevelop and increase the diversity of staff and todevelop research excellence in all the schools. Acomprehensive recruitment and selectionprocedure has been adopted to ensureappropriate appointments, and new academicstaff participate in a three-year programme ofsupport, incorporating mentoring on theirresearch activities, research training and theundertaking of the Postgraduate Certificate inTeaching in Higher Education (PGCTHE). Allcandidates for promotion are expected todemonstrate effectiveness and excellence as ateacher. Excellence in teaching has also beenrewarded and good practice disseminated bythe creation of designated principal lecturerposts specialising in teaching and learning andacademic fellowships for one-year projects inlearning and teaching. In addition a scheme ofteaching fellowships, associate teachingfellowships and learning support fellowships hasbeen introduced to support pedagogicdevelopment in areas that contribute to thestrategic objectives of the University.

220 Schools are required to maintain staffdevelopment plans under a University policy forPersonal and Professional Development andstaff development needs are identified as partof the process of annual personal developmentand review. As well as an annual programme ofstaff development workshops, the OxfordCentre for Staff Learning and Developmentprovides the PGCTHE for new staff, a SeniorStaff Development Programme for senior staffwith managerial responsibilities and a CoreManagement and Leadership Programme forstaff aspiring to become senior managers.

The outcomes of the discipline audit trails

Business and management 221 The DAT covered four undergraduateprogrammes (BA (Hons) Business andManagement, BA (Hons) International BusinessManagement, BA (Hons) Business Logistics andBA (Hons) Business Innovation and Enterpriseand the MSc Business and Enterprise.

222 From discussions with staff and students,the study of assessed student work and thedocumentation reviewed, the audit teamformed the view that the standard ofachievement in the business and managementarea is appropriate to the titles of the awards.The programme aims and the learningoutcomes at both undergraduate and master'slevels are in line with The framework for highereducation qualifications in England, Wales andNorthern Ireland (FHEQ). Appropriate referencehas been made to the expectations of thebenchmark statements in this area.

223 Students felt well supported by the staff,who were accessible and approachable. Thelearning resources were well used and effectiveand despite initial problems withsemesterisation the students felt confident intheir ability to achieve and progress in theirprogrammes. They felt well prepared for theirfuture careers. The audit team concluded thatthe quality of the learning opportunities offeredin the business and management area wasappropriate to the programmes of study.

English studies224 The DAT covered two undergraduateprogrammes, the BA Single Honours, EnglishStudies and the BA Combined Honours, EnglishStudies, together with a master's levelprogramme, the PostgraduateCertificate/Postgraduate Diploma/MA, Modernand Contemporary Poetry. From discussionswith staff and students, the study of assessedstudent work and the documentation reviewed,the audit team formed the view that thestandard of achievement in the English subjectarea was appropriate to the titles of the awards.


page 47

The programme aims and the learningoutcomes at both undergraduate and master'slevels are in line with the FHEQ. Appropriatereference has been made to the expectations ofthe benchmark statements in this area.

225 Sound procedures for the assurance ofquality and standards were confirmed, with anappropriate management of studentassessment. Students appreciated the positivesupport of staff and the quality of informationand advice made available in studenthandbooks. Some concern was expressedregarding the availability of secondary text.

226 The audit team concluded that the qualityof the learning opportunities offered in theEnglish studies subject area was appropriate tothe programmes of study.

Construction management and urban design227 The DAT covered two undergraduateprogrammes: BSc (Hons) Building and BSc(Hons) Construction Management, togetherwith a master's programme, the MA in UrbanDesign. From its discussions with staff andstudents, the study of assessed student workand the documentation reviewed, the auditteam formed the view that the standard ofachievement in the subject area wasappropriate to the titles of the awards.

228 A wide range of assessment methods isused including a significant amount of groupwork. While students recognised the importanceof group work in the construction industry theyexpressed some concern that individualcontributions were not always recognised.Students expressed particular appreciation of thecomprehensive student handbook and theindustrial placement opportunity which preparedthem for a professional career. Quality assuranceof the programmes was generally judged to beoperating appropriately although the audit teamconsidered that the process would be enhancedby a more detailed analysis of statistical data.

229 The audit team concluded that the qualityof the learning opportunities offered in the BuiltEnvironment subject area was appropriate tothe programmes of study.

Mechanical engineering230 From its study of students' assessed work,and from its discussions with staff and students,the audit team formed the view that thestandard of student achievement in themechanical engineering discipline area wasappropriate to the title of the awards and theirlocation within the FHEQ. The programmespecifications set out appropriate educationalaims and learning outcomes and link them toteaching, learning and assessment. Theprogramme specifications reflect expectationsin the subject benchmark statements.

231 Student evaluation of the programmes ishighly supportive and very positive, particularlyin relation to placements, student support andguidance. Learning resources were felt to bewholly appropriate and accessible.

232 The audit team concluded that the quality ofteaching and the learning opportunities offeredin the Mechanical Engineering subject area wasappropriate to the programmes of study.

Philosophy and religious studies233 The DAT covered the following programmes:BA Combined Honours in Philosophy; BA Combined Honours in Religious Studies.

234 From discussions with students and staffand its study of assessed work, the audit teamformed the view that the standard of studentachievement was appropriate to the titles of theawards and their location in the FHEQ. Studentevaluation of the programmes was positive andthere was evidence that staff were accessible tostudents and responsive to feedback fromthem. A variety of assessment strategies iseffectively deployed to enable students toachieve the learning outcomes in theprogramme specifications.

235 The audit team concluded that the qualityof the learning opportunities available to studentsin the religious studies and philosophy area wasappropriate to the programmes of study.


page 48

The institution's use of the AcademicInfrastructure

236 It was evident that institutional policy was informed and guided by the AcademicInfrastructure. Within the quality assurancesystems were procedures for embeddingelements of the Code of practice for theassurance of academic quality and standards inhigher education, published by QAA within theinstitutional procedures which ensuredcompliance at the programme level. The auditteam saw sufficient evidence that subject staffwere aware of and made reference to the Code.

237 The SED stated that the University has aclearly defined framework of academicqualifications based on academic level andacademic credit and that it welcomed thedevelopment of the FHEQ and expects thequalification descriptors to be used as part ofthe approval and review process. The auditteam confirmed that awards were inline withthe level descriptors. It was noted that theUniversity had defined its undergraduateawards of consisting of two parts containingthe three level descriptors. This approach wasclear to all students and staff.

238 Through its work the audit team foundthat subject areas had fully embraced subjectbenchmark statements. Programmespecifications are fully embedded and haveassisted in curriculum redesign anddevelopment. It also noted that the flexibilityoffered in terms of programme specificationshad led to some diversity in approach andstandards of documentation. The University is now moving towards publishing programme specifications on their website. The team also noted that existing reviewprocedures did not make explicit reference tosubject benchmark statements.

239 Overall, evidence suggested that theUniversity had engaged meaningfully with theAcademic Infrastructure.

The utility of the SED as an illustrationof the institution's capacity to reflectupon its own strengths andlimitations, and to act on these toenhance quality and standards

240 The SED provided a useful description ofthe University, the main processes in relation toquality and standards, a statement of the recentchanges that have taken place in the Universityand a description of the supporting services.The University also provided a set of supportingreferences which formed the basis of the auditteam's work. The team pursued additionalinformation on the areas described in the SED,collecting details on the variation in practice inthe schools within the University from the DATsand the discipline self-evaluation documents.Overall the SED offered the team an initialstatement of the view of academic standardsand quality as seen from the centre of theUniversity but was sometimes lacking inevaluation of these processes, or details of theiroperation in the schools.

Commentary on the institution’sintentions for the enhancement ofquality and standards

241 The University has introduced a number ofchanges to enhance the quality of itsprogrammes and the whole studentexperience. New developments in learning andteaching practice include conferences and apeer reviewed e-journal. It has also introduceda Quality and Customer Care policy withinwhich is included the enhancement of theservice provided to students.

242 A major change which took effect fromSeptember 2004 was the re-division of theteaching year from terms to semestersaccompanied by a shortening of the academicyear so that it finishes just after the middle ofMay. A senior group was established to managethe process and began its work in 2002. Despitecareful planning and attention to detail, some ofthe problems have been difficult to deal with anda number of problems have arisen for students.These continue to be dealt with as they arise.


page 49

243 The plan for the revised academic yearincluded the intention to complete semesterone before Christmas. This has led to difficultieswith the assessment period and somedisruption to the final week of teaching. Theaudit team recognises that issues such as thesehave the potential to affect adversely thequality of the student experience, and theUniversity is therefore advised to continue itsprogramme of optimising the academic year toenable such problems to be eliminated.

244 The several problems which have arisenduring the first year of semesterisation haveaffected a number of students in various ways.It is important that the overall learningexperience of current and future students isdiminished as little as possible and theUniversity is advised to continue to address thedeficiencies which have been identified.

245 Students have experienced problems withthe timing and load of both formative andsummative assessment. The audit team advisesthe University that continuing attention shouldbe paid to ensuring that assessment loads onstudents are suitably spaced in time and are notallowed to rise above an acceptable level. Thetimetabling of student contact hours has alsoproved to be difficult for some full-time andsome part-time students. It is desirable that thisalso be the subject of continuing attentionwithin programme teams.

The reliability of information

246 The students' view is that generally theinformation provided is reliable accurate andfair. However, at the time of the audit theUniversity was undergoing substantial and rapidchange and as a result information quicklybecame out of date.

247 Students have continuous and immediateaccess to key individual information relevant totheir studies through the on-line PIP developedby the University. The PIP enables students tomanage their personal and programme-relatedinformation and to see their timetable,examination and assessment schedules. Itprovides a virtual link with personal tutors and

to the library and support services. Staff andstudents were enthusiastic about the benefits ofPIP and it was generally considered by staff andstudents to be an excellent system.

248 The University was aware of the currentissues regarding reliability of information andhad made substantial efforts to ensure thatstudents were properly informed by a variety ofmethods, and had ensured that the informationon the pages was current. Nevertheless somestudents did experience difficulties which werecompounded by staff not always having themost up-to-date information to give to students.

249 The University has put in place thenecessary procedures to ensure that publicinformation is placed in the Higher Educationand Research Opportunities in the UK website.Preparations are well advanced to support thepublication of programme specifications,summaries from external examiners' reports, andthe University's internal periodic reviews ofsubject areas. The University's compliance withthe requirements of HEFCE's document 03/51,Information on quality and standards in highereducation: Final guidance, is monitored throughthe QSC. Other information is verified either bythe Directorate of Corporate Affairs or in schools.

Features of good practice

250 The audit team identified the followingareas as being good practice:

i the quality of support for postgraduateresearch students (paragraphs 116 to 118)

ii the on-line PIP developed by theUniversity for its students (paragraphs112, 141, 193)

iii the level of accessibility of staff withinschools and their support for students,underpinned by the information providedto staff by Student Services (paragraphs113 to 115, 144, 156, 171, 186)

iv the series of themed audits undertaken bythe APQU (paragraph 55).


page 50

Recommendations for action

251 It would be advisable for the University to:

i strengthen the quality assurance processesat institutional level in order to secure asufficiently effective oversight by AB andits committees of their operation in theschools (paragraphs 39, 201)

ii continue to address the identifieddeficiencies in the quality of learningexperienced by some students in the firstyear of semesterisation (paragraphs 44 to 49)

iii develop and publish a plan for thesemesterised academic year which ensuresoptimal coordination and operation(paragraphs 45-46, 142, 172)

iv review its assessment procedures withparticular attention to consistency, timing and load on students (paragraphs 45, 47, 172).

252 It would be desirable for the University to:

i make more effective use of the annualreview process and develop further itsformal systems for the dissemination ofgood practice across the institution(paragraphs 53, 205-206)

ii work more closely with the officers of theSU in order to improve the utility ofstudent involvement at the institutionallevel (paragraphs 75-76)

ii continue to develop a more strategicapproach to the use and analysis ofstatistical data within review and decision-making processes (paragraphs 87,147, 159, 169).


page 51

Appendix

Oxford Brookes University's response to the audit report

The University welcomes the Agency's finding that broad confidence can be placed in thesoundness of the University's current and future management of the quality of its programmes andof its capacity to manage effectively the academic standards of its awards.

We particularly welcome the recognition of the quality of support to postgraduate research studentsand the efforts made by all the staff involved to provide such an excellent experience.

The University is also pleased to see that the work that has gone into providing such an effectiveand well-used Personal Information Portal system has been duly recognised.

We are also pleased to see that the time and effort given by staff to support students, both withinschools and through Student Services, which has always been recognised as first class by the QAA duringSubject Review and the Developmental Engagements continues to be appreciated.

Finally the University welcomes the credit given by the audit team to the work of the APQU inrelation to the themed audits that have been carried out and are planned for the future.

The University will be taking a variety of actions to address the recommendations provided in the report.

The terms of reference of the Academic Board and its main committees are already being reviewed toensure there is greater clarity of their individual responsibilities, while continuing with the principlethat the Quality and Standards Committee is the key University committee with responsibility for thequality and academic standards of its programmes of study.

The University accepts that there are further steps that it can take to improve the dissemination of goodpractice more widely and will be producing proposals to address this matter.

The University confirms that it has carried out the review of the first year of semesterisation and hasan action plan to address the various issues that have been highlighted throughout the year. It isstrongly committed to ensuring that the quality of the student experience is maintained andenhanced and will be making any changes it sees as necessary to the implementation of the newsemesterised academic year. It will continue to monitor closely the views of students and staff toidentify further enhancements.

The University values highly the role the Students' Union plays in representing the students' viewsand will be working closely with it to enhance student involvement in University processes at alllevels. The University has been working with the officers of the Students Union on changes to theconstitution which are designed to improve participation and accountability within the Union.


page 52

RG

170 11/05

rg170 oxford brookes uni - dera.ioe.ac.uk · the audit process institutional audits are carried out...

Documents