internal quality audit handbook - quality – safety · pdf filekeele university internal...

Academic Affairs

Quality Assurance and Standards Division

INTERNALQUALITY

AUDITHANDBOOK

2006 edition

KEELE UNIVERSITYINTERNAL QUALITY AUDIT

2

Contents

Page

Foreword 3

Section 1: Overview of Internal Quality Audit 5

Section 2: The audit team, their roles and responsibilities 10

Section 3: The external adviser 13

Section 4: The roles and responsibilities of committees and officersof the University including executive Deans of Faculties 15

Section 5: The timeline 17

Section 6: Preparing for Internal Quality Audit 19

Section 7: The self-evaluation portfolio (SEP) 20

Section 8: The Internal Quality Audit Report and Summary 22

Section 9: Sample timetable / sample agenda for an internal qualityaudit event 24

Section 10: Aspects of provision - an aide-memoire for Schoolsand auditors 26

Section 11: Report form for use by the External Adviser 30


3

Foreword

1. Keele University has had processes for reviewing aspects of the quality ofteaching in since the 1980s. Internal Quality Audit (IQA) was first approved bySenate in 1994-95 and the previous fundamental revision of the IQA processwas in 1999. The IQA process in that era was strongly influenced by themethods for teaching quality assessment / subject review developed byHEFCE and subsequently the QAA for use in England in 1996 and 1998.Minor updates of the 1999 edition of the IQA Handbook were made from timeto time.

2. This fundamental revision of Keele University’s process of periodic InternalQuality Audit takes account of:

The aim, principles and framework of the university’s quality assurancepolicy, set out in the Academic Quality and Standards Manual andrelated codes of practice and other documents, all available athttp://www.keele.ac.uk/depts/aa/qao/qamanual/index.htm

The UK Higher Education code of practice for the assurance ofacademic quality and standards in higher education, comprising theframework for higher education qualifications in England, Wales andNorthern Ireland (FHEQ), subject benchmark statements andprogramme specifications, all available athttp://www.qaa.ac.uk/academicinfrastructure/default.asp

The repeated and explicit commendations of IQA in reports of externalquality reviews, most recently in reports from DevelopmentalEngagements, and Institutional Audit 2004.

3. This revision retains peer review as the basic approach to Internal QualityAudit and the School as the normal unit of audit.

4. This revision is gives effect to:

A more thematic approach to internal audit.

The expectation that Keele’s policies and procedures for the assuranceand enhancement of quality and standards (AQSM etc) are embeddedin academic programmes throughout the university. It should befeasible to make more use of routine documentation, with lessemphasis on specially commissioned lengthy documents such as a SelfEvaluation Document.

The expectation that Schools’ responses to the QAA academicinfrastructure and other external points of reference are embedded intheir academic programmes.


4

The flexibility required to audit Schools which are fewer in number thanin the past but which vary considerably in their complexity, in theirrange and variety of their programmes, and in their stage ofdevelopment.

Recognition of the role and authority of the executive Deans ofFaculties, and changes to the mechanics of quality assurance broughtabout by Keele 2006.

The introduction of a Summary Report, required of the university forexternal publication as part of its suite of Teaching Quality Information(TQI), and an accelerated timescale for the approval of reports in orderto meet the requirements of TQI

Internal (only) publication of the full audit report as a contributor to the enhancement of quality and standards

Professor R C J CocksPro Vice-Chancellor15th December 2005


5

Section 1: Overview of Internal Quality Audit

1.1 Internal Quality Audit is an important element in the University’s strategy andprocedures for assuring the quality of learning opportunities provided tostudents and the standards of programmes and awards. This strategy andprocedures are set out in the Academic Quality and Standards Manual(AQSM). This manual forms part of the AQSM.

1.2 It is an explicit aspect of the University’s quality assurance strategy thatSchools take responsibility for assuring their quality and standards, as set outin section 5 of the AQSM. The university is required to have processesthrough which confidence is established at the level of the institution in thequality of learning and teaching provided by Schools, in the standards ofachievement that are set and maintained by Schools, and in the carrying outof university policies directed to these ends. These processes include theapproval of new courses and modules and changes to these, scrutiny ofannual reports of external examiners, annual monitoring visits by the QualityAssurance Office to Schools, and periodic Internal Quality Audit.

1.3 Normally, each School has an Internal Quality Audit on a five yearly cycle, butQuality Assurance Committee may order an Audit at any time, for instance inresponse to concerns raised by another aspect of the University’s qualityassurance process. The five-year cycle for internal quality audit complementsthe three-year cycle of Programme Review (aka Triennial Course Review)carried out by Schools themselves and the annual cycles of external examinerreports and the annual monitoring visit.

1.4 Important aspects of internal quality audit are that it is transparent and that it isessentially a process of peer review, involving participants from within theFaculty of which the School is a part, from the wider University, and from thediscipline(s) nationally.

1.5 The focus of the internal quality audit is to be

(1) the processes used by the School for the maintenance andenhancement of the quality of education and the standards ofachievement, with sufficient examination of the product of theseactivities to establish confidence that the processes are effective andthat the quality and standards are sustainable, and

(2) the response of Schools to the QAA codes of practice andcompliance with Keele University policies and regulations, including theprovisions of the Academic Quality and Standards Manual.

1.6 The normal unit of audit to be the School. If the breadth and complexity of theunit’s educational provision require it, the Chair of Quality AssuranceCommittee and/or Director of Quality Assurance in consultation, may increasethe size of the audit team, the number of External Advisers, the number of


6

meetings, and the duration of the event.

1.7 The audit team will be determined by the Director of Quality Assurance inconsultation with the Chair of the Quality Assurance Committee and the Deanof the Faculty. The normal composition of an audit team is (see section 2 onthe roles of members of audit teams):

The audit chair, a senior member of teaching staff of the university withwide experience of teaching and the management of teaching andwhere possible with experience of Internal Quality Audit.

Two auditors, both teaching staff of the university with experience of themanagement of teaching, one of these from the Faculty of which theSchool is a part but not from the School under audit.

An external adviser of national standing in one or more of thedisciplines covered by the audit appointed by the Director of QualityAssurance having consulted the Head of School and Faculty Dean. Seealso section 3.

The audit secretary, from the Quality Assurance Office, usually aDeputy Director of Quality Assurance.

1.8 The most important working set of documents for the internal quality audit willbe a Self-Evaluation Portfolio. This will comprise the following (see section 7for greater detail):

A framework summary

Evaluation of the Strengths and Weaknesses of the School and theOpportunities and Threats facing it.

A statement from the Dean of the Faculty (Optional)

A student submission, either in addition to or in place of a face-to-facemeeting with students. (Optional)

(Note: Normally, the items listed above will be the only specially commissioneddocuments, the remainder being School quality assurance documentation whichshould be in routine use.)

Learning and Teaching Strategy

Most recent two Annual Monitoring Visit reports

For each of the undergraduate programmes, taught postgraduate


7

programmes, and research training schemes within the scope of theaudit: Programme specification and course regulations, most recenttriennial course review report, most recent annual course review report,most recent two years’ reports from external examiners and Schools’responses to these, current student handbooks, and, where applicable,the most recent report from accreditation by Professional or StatutoryBodies.

1.9 Further to #1.8, where it is practicable, IQA's are to be scheduled so that usemay be made of documentation prepared for accreditation by Professional andStatutory Bodies or for similar events.

1.10 Normally, if it is feasible, a meeting with a representative group of taughtstudents will be held in advance of the audit event. When this is not possible,the students to be invited to make a written submission, preferably a collectiveone. Auditors will expect to meet with student academic representatives atlunch on audit day.

1.11 Having received the SEP and other documents, and guided by the AuditSecretary, the audit team (without the External Adviser) will agree their lines ofenquiry. This may involve a meeting or communication by e-mail or both. Theaudit chair and audit secretary will formulate an agenda for the audit day, inconsultation with the Head of School. The audit secretary will makearrangements for meeting rooms and refreshments in collaboration with theSchool.

1.12 On the day of the audit, the Audit Team (including the External Adviser) willhave a series of meetings with appropriate staff and students from the Schooland private meetings in order to pursue their lines of enquiry and to gather andevaluate evidence.

1.13 The Auditors will prepare a draft report based on the Audit, summarising theiranalyses and conclusions and recommendations under the following aspectheadings:

aims, intended learning outcomes and the curriculateaching and learning assessment student progression and student support learning resourcesenhancement of quality and standards

1.14 Within four weeks of the audit event, the audit secretary will send the draftreport and the summary for publication to the Head of School for correction oferrors of fact or of interpretation. The School will have two weeks in which tonotify the Quality Assurance Office of any errors of fact or of interpretation.

1.15 Within eight weeks of the audit event, the final version of the audit report and


8

of the summary for publication will be sent to the Head of School, the Dean ofFaculty, and the Chair of Quality Assurance Committee.

1.16 The School will have a period of four weeks to develop an action plan inresponse to the Audit Report, to sustain strengths and remedy weaknesses.By the end of this period, the School must seek and gain the approval of theDean of the Faculty for this action plan and submit the action plan to theQuality Assurance Office (through the Audit Secretary) so that it can beconsidered by Quality Assurance Committee.

1.17 The Audit Secretary will check the action plan for completeness, raising anynecessary questions with the School and Dean concerned, before submissionto Quality Assurance Committee in the interests of efficient use of theCommittee’s time. As soon as possible thereafter, the audit report, action planand the summary for publication will be considered by Quality AssuranceCommittee.

1.18 Quality Assurance Committee may accept the findings of the internal qualityaudit in whole or in part and/or may require follow-up action by the Chair ofQuality Assurance Committee or by the Director of Quality Assurance. Follow-up actions might include a repeat audit or partial repeat audit orrecommendations to the Senate.

1.19 When Quality Assurance Committee has considered the audit report andaction plan, the report with the action plan as an annex, will be published onthe university’s intranet in order to foster enhancement through thedissemination of good practice. In exceptional circumstances, Heads ofSchool with the support of the Dean may seek exemption from this in whole orin part. Applications for exemption will be decided by Quality AssuranceCommittee or by its Chair or by the Director of Quality Assurance.

1.20 As part of its consideration of the audit report and action plan, QualityAssurance Committee will either authorise the summary report for publicationon the TQI website or withhold publication pending some action.

1.21 12 months after the action plan was submitted, or at such later or earlier timeas may be determined by the Director of Quality Assurance taking intoaccount the university diary of meetings, the Head of School will submit a briefreport of progress on implementing the action plan to the Dean of the Facultyand to the Quality Assurance Office for report to the Quality AssuranceCommittee.

1.22 Confidentiality of documents and discussions.

Auditors will treat all documents provided by the School for the Audit, and alldiscussions and notes made at meetings, as confidential to the team and tothe School. The report of the External Adviser will be made available only tothe auditors and will be confidential to them and the Director of Quality


9

Assurance.

The written student submission (if any) will usually be made available both tothe School and to the audit team. Exceptionally, the School’s studentacademic representatives or an officer of the Students Union may request thatthe written student submission be withheld from the School; in such a case,the Director of Quality Assurance and the Chair of the Quality AssuranceCommittee will adjudicate.

The Audit Report and Summary Report will be written in such a way as torespect the confidentiality of the written or oral contributions of any individualstaff member or student.


10

Section 2: The audit team, their roles and responsibilities

2.1 There will be a panel of auditors, teaching staff of the university withexperience of the management of teaching. The Director of Quality Assurancewill appoint auditors to the panel on the advice of the Chair of QualityAssurance Committee, Deans of Faculties and Heads of School, asappropriate. It is expected that auditors will participate in a maximum of twoaudits per year.

2.2 Auditors designated as chairs of audit teams will be senior members ofteaching staff of the university, with wide experience of teaching and themanagement of teaching* and, where possible, with prior experience ofInternal Quality Audit.

*e.g. as Director of Study, Programme Leader, or Head of School

2.3 Novice auditors will be trained through preparatory documentation, briefingsessions and observation at part of an audit event. Experienced auditors willbe updated from time to time by means to be determined by the Director ofQuality Assurance.

2.4 The team for the Internal Quality Audit of a School will be determined by theDirector of Quality Assurance in consultation with the Chair of the QualityAssurance Committee and the Dean of the Faculty.

2.5 The normal unit of audit to be the School. If the breadth and complexity of theunit’s educational provision require it, the Chair of Quality AssuranceCommittee and/or Director of Quality Assurance in consultation, may increasethe size of the audit team, the number of External Advisers, the number ofmeetings, and the duration of the event.

2.6 The normal composition of an audit team is:

The audit chair (see 2.1)

Two auditors, both teaching staff of the university with experience of themanagement of teaching, one of these from the Faculty of which theSchool is a part but not from the School under audit.

An external adviser of national standing in one or more of thedisciplines covered by the audit appointed by the Director of QualityAssurance having consulted the Head of School and Faculty Dean.(See also section 3).

The audit secretary, from the Quality Assurance Office, usually aDeputy Director of Quality Assurance.


11

2.7 The audit chair is to lead the audit team in formulating their lines of enquiry,participating if possible in the meeting with students (if there is one), chair themajority of the meetings on the day of the audit, and sign off the draft auditreport. The audit chair will also share the duties of the other auditors.

2.4 The role of the auditors is

• to scrutinise the Self Evaluation Portfolio and to develop lines of enquirybased on the aspects of provision listed in section 10.

• each to take responsibility for collating the evidence, analyses andevaluations of two of the six aspects, noting that all auditors havecollective responsibility for the evaluations

• to formulate an agenda for the audit event in order to pursue these linesof enquiry

• to identify any additional documentary evidence needed • to meet with students studying on the programmes under audit in order

to obtain their opinion of the quality of their learning and the teachingand guidance that they receive from the School

• to participate in meetings with staff and students on the day of the audit• to participate in meetings with representatives of Public, Statutory or

Regulatory Bodies, commissioning agencies or employers asappropriate to the programmes under audit

• to prepare a draft report based on the Audit, summarising theiranalyses and conclusions and recommendations under the aspectheading given in section 10, each auditor taking responsibility fordrafting an account of two of the aspects, including the evidence,analysis and judgement, and for identifying good practices orweaknesses that are to be highlighted in the report summary

• to comment on the draft audit report as a whole.

2.5 The role of the External Adviser is set out in section 3.

2.6 The role of the audit secretary is

• to liaise with the Chair of Quality Assurance Committee, the Director ofQuality Assurance, the Dean of the Faculty and the Head of Schoolunder audit in clarifying the scope of the Audit.

• to assist the Director of Quality Assurance in appointing the Audit Teamand External Adviser.

• to advise the Head of School (or representative) on preparing forInternal Quality Audit.

• to make arrangements for the meeting with students (if any) and/or forwritten submission(s) by students of the School under Audit.

• to consult on and prepare the agenda and timetable for the Auditevent(s)

• to coordinate physical arrangements and facilities and catering for theAudit.

• to brief and advise the Auditors and External Adviser as required.


12

• to coordinate the allocation of duties between Auditors, and preparatorymeeting(s) and/or discussions.

• to be responsible for arrangements on Audit day, includingaccommodation for and entertainment of the External Adviser asrequired.

• to make a written record of the meeting with students (if held), ofmeetings and discussions of the Audit Team, and of the Audit event.

• to collect, integrate and edit the contributions from individual auditorsinto the draft report and to be responsible for the format of the reportand uniformity of style.

• to write the Summary Report• to make arrangements with the Head of School for checking the Draft

Audit Report and Draft Summary Report for factual accuracy, and tomake appropriate alterations to these

• to send the Audit Report and Summary Report to the Chair of QualityAssurance Committee, the Director of Quality Assurance, the Dean ofthe Faculty and the Head of School.

2.7 All members will carefully observe the confidentiality of all documents anddiscussions.


13

Section 3: The External Adviser

3.1 The External Adviser is a practitioner of national standing in one or more ofthe disciplines covered by the audit and is appointed by the Director of QualityAssurance having consulted the Head of School and the Faculty Dean.

3.2 It is expected that between them the Head of School and Faculty Dean willnominate three possible External Advisers, ideally from Schools at other UKuniversities with comparable scope to the School under audit and who canprovide an independent evaluation of the standards of achievement and of thequality of learning and teaching in the School under audit. Exceptionally, morethan one External Adviser may be appointed if it is not possible for one toadvise the audit team on the breadth of programmes under audit.

3.3 The criteria for appointment as External Adviser are

• wide experience and competence at a senior level in a teachingSchool at another UK university,

• a practitioner in at least one of the subject areas under audit withnational credibility within the general subject area,

• familiarity with academic standards within UK universities,• sufficient standing within the at least one of the disciplines to

speak authoritatively on standards, and on course design andcontent,

• familiarity with the standards of achievement to be expected ofstudents in the programmes under audit

Normally, then, an External Adviser is likely to be a senior academic, usuallyholding an appointments as Senior Lecturer or above in a School coveringsubjects identical with or similar to those being audited, in a UK universityoffering courses comparable to those offered at Keele.

It is a requirement that an External Adviser should not

• have been a student or a member of the academic staff at Keelein the previous six years;

• be subject to any reciprocal arrangement or conflict of interest.

All of these criteria may be relaxed, with the agreement of the Director ofQuality Assurance, but only in exceptional circumstances (e.g. in cases ofextreme urgency or extreme difficulty in finding an External Adviser with asuitable breadth of expertise or in a in a subject area where there is a verylimited number of potential External Advisors).

3.4 A fee and expenses are paid to the External Adviser.

3.5 The university has included External Advisers in Internal Quality Audit teamssince 1997. Before then, audit teams had had recourse to External


14

Examiners’ reports for the School under audit. The change was made toenhance the reliability and independence of the external assessment ofacademic standards. Although audit teams do, of course, consider a School’sexternal examiners’ reports, the contribution made by the external adviser tothe audit process has been valued, both by audit teams and by Schools.

3.6 The External Adviser is not an auditor and usually is not involved in writing theaudit report, but participates in meetings and produces a brief report at theend of the audit visit (on a pro forma for that purpose) to aid the team in theproduction of the Audit Report. The External Adviser may request that a copyof her/his report be sent to the School, which may if it wishes comment on it.The External Adviser may bring any serious concern about academicstandards directly to the notice of the Vice-Chancellor.

3.7 The precise arrangements for the involvement of the external adviser mayvary from one audit to another. The adviser may choose to attend all themeetings on the day of the audit visit; occasionally, because of the travellingdistance involved, he/she attends only some of the meetings. Audit teamsmay have different arrangements for the level of involvement they expect froman adviser during the day: this is a matter for agreement between the Directorof Quality Assurance, the audit chair and the External Adviser. The attentionof the adviser is drawn to the need to keep to time during the scheduledmeetings of the audit visit.

3.8 As well as giving additional credibility to the audit process, External Advisersare able to offer a comparative perspective on a School’s activities. In thepast this has been of particular use in such areas as the curriculum,programme specifications, module descriptions, course handbooks etc. TheExternal Adviser will be supplied by the audit secretary before the audit visitwith a selection from the School’s briefing materials, including the SelfEvaluation Portfolio.

3.9 The adviser would normally expect to gain an overview of course contentthrough consideration of bibliographies and assessment tasks. A post-lunchmeeting is often arranged (unless the adviser indicates otherwise) betweenthe External Adviser, one member of the audit team, and colleagues fromKeele Information Services. The School may choose to ask its library link-person (or appropriate member of staff) to accompany the adviser on that visit.

3.10 The External Adviser is invited to stay overnight before the Audit event asguest of the Director of Quality Assurance. At the discretion of the Chair ofQuality Assurance, the External Advisor is invited to dine with one or more of:a Pro-Vice-Chancellor, Dean of the Faculty which includes the School,Director of Quality Assurance (or Deputy), Chair of the Audit Team.


15

Section 4: The roles and responsibilities of committees and officers of theUniversity including executive Deans of Faculties

4.1 Quality Assurance Committee

Quality Assurance Committee approves

• the rolling programme of periodic Internal Quality Audit, • receives and considers audit reports, • decides on actions following audit, • authorises publication of reports and summary reports, and • submits its findings from Internal Quality Audit to the Senate.

4.2 Chair of Quality Assurance Committee

The Chair of Quality Assurance Committee

• is consulted on the composition of audit teams• receives a copy of each Internal Quality Audit report as soon as

it has been checked for factual accuracy• if required and with the Director of Quality Assurance,

adjudicates on applications to restrict the Student Submission tothe audit team,

• if required and with the Director of Quality Assurance,adjudicates on applications to withhold or restrict publication ofthe Summary Report or Audit Report, and

• following consideration of an audit report by Quality AssuranceCommittee, determination of actions to be taken (if any) followingaudit.

4.3 Director of Quality Assurance

The Director of Quality Assurance has operational responsibility for theplanning and conduct of Internal Quality Audits, including

• the appointment and training of the panel of auditors, • the appointment of auditors to teams in consultation with the

Faculty Dean,• liaison with Schools under audit, • if required and with the Chair of Quality Assurance Committee,

adjudicates on applications to restrict the Student Submission tothe audit team,

• reporting of audit outcomes to Quality Assurance Committee,• if required and with the Chair of Quality Assurance Committee,

adjudicates on applications to withhold or restrict publication ofthe Summary Report or Audit Report, and

• actions following audit (if any).


16

4.4 Executive Deans of Faculties

Deans of Faculties

• are consulted on the appointment of auditors to the panel,• are consulted on the composition of audit teams for Schools in

their Faculty including the External Adviser, • have the opportunity to contribute a statement to the Self

Evaluation Portfolio for Schools in their Faculty under audit,• receive a copy of each Internal Quality Audit report on a School

in their Faculty as soon as it has been checked for factualaccuracy, and

• must approve the Action Plan devised by Schools in their Facultyin response to Internal Quality Audit Reports before submissionto Quality Assurance Committee.

Deans of Faculties are not eligible to be auditors.


17

Section 5: The time line

The time line is as follows

12 weeks before event: Nomination and appointment of External AdviserAppointment of audit teamDesignation of audit chairDesignation of audit secretary

5 weeks before event: Submission of Self Evaluation Portfolio to theQuality Assurance Office

4 weeks before event: Auditors meet (or have virtual meeting) to agreelines of enquiry and to identify additionaldocuments (if any) to be requested from the Schoolor to be provided by the School on the day of theaudit event.

3 weeks before event: Audit chair and audit secretary agree agenda forthe audit event and liaise with the Head of Schoolover this and staff to meet auditors (see section 9).

2/3 weeks before event: Auditors’ meeting with students (if practicable)

3/2/1 weeks before event: Audit secretary liaises with Head of Schoolor representative over logisticalarrangements (rooms, refreshments, etc)

The sequence of events following the audit event will be as follows:

Event + 4: Draft report and summary for publication sent to School forcorrection of factual errors.

Event + 6 weeks: Factual errors submitted to Quality Assurance Office.

Event + 8 weeks: Final version of report to Head of School, Dean of Faculty,and Chair of Quality Assurance Committee

Event + 12 weeks: Action plan agreed by School and Dean of Facultysubmitted to Quality Assurance Committee.

The Audit Secretary will check the Action Plan and Follow-up Report forcompleteness, raising any necessary questions with the School/School concerned, before submission to Quality Assurance Committee in theinterests of efficient use of the Committee’s time.

As soon as possible thereafter, Audit Report and Action Plan and summary for


18

publication considered by Quality Assurance Committee

Full report and action plan published internally on the web in order to fosterenhancement through the dissemination of good practice. In exceptionalcircumstances, Heads of School may seek exemption from this in whole or inpart. Applications for exemption to be decided by Quality AssuranceCommittee or by its Chair or by the Director of Quality Assurance.

Audit Summary published to TQI website

QAC meeting plus 12 months, Progress report from School and Dean onimplementation of action plan


19

Section 6: Preparing for Internal Quality Audit - requirements and advice

The following is a suggested brief checklist for Schools preparing for Internal QualityAudit. But note that it is not exhaustive. It is expected that key staff in Schoolspreparing for Audit will develop a detailed knowledge of this Handbook.

See also the Time line for Internal Quality Audit (section 5)

6.1 It is generally helpful if a member of staff of the School has prior experience asan Internal Quality Auditor or as an auditor/reviewer for the Quality AssuranceAgency. Such a colleague will be able to advise on the development of the SelfEvaluation Portfolio and on other matters related to the Audit.

6.2 Clarification of the scope of the Internal Quality Audit with the Director of QualityAssurance.

6.3 Nomination of candidates for appointment as External Adviser (see section 3)

6.4 Development and submission of the Self Evaluation Portfolio (see section 7),seeking advice from the Audit Secretary as required.

6.5 Nomination of students to meet Auditors prior to audit event

6.6 Selection of Student Academic Representatives to meet auditors and ExternalAdviser on the day of the audit

6.7 Liaison with the Audit Secretary on the agenda and timetable for the audit event,and the selection of staff to meet auditors at the various times/meetings

6.8 Liaison with the Audit Secretary on logistical arrangements: room(s) for meetingsand viewing documents, tour of facilities, and room for working lunch and otherrefreshments. Note that usually the Audit Secretary will make arrangements withKFM for the supply of lunch and other refreshments.

6.9 Selection of supporting documentation / wider resources to be available to theauditors in the School on or before audit day. These are dependent on the lines ofenquiry developed by the Audit Team but are likely to include routine documentationfrom the School’s quality assurance processes including reports from annual andtriennial course reviews, minutes from Learning and Teaching Committee orequivalent, from course committees or equivalent, from Boards of Examiners andfrom Staff-Student Liaison Committees, reports from External Examiners andresponses, and where applicable from accreditation by Professional or StatutoryBodies.

6.10 Designation of a member of staff of the School (academic or support staff asappropriate) to liaise with the Audit Secretary before, during and after the AuditEvent.


20

Section 7: The self-evaluation portfolio (SEP)

7.1 Schools are strongly encouraged where it is practicable to do so to useexisting documents as components of the self-evaluation portfolio (SEP).

7.2 Audit teams will respect the confidentiality of the documentation provided bythe School, and will use information derived from the portfolio only for thepurposes of Internal Quality Audit.

7.3 The Self-Evaluation Portfolio will comprise:

(1) A contents list.

If validation or accreditation submissions or other such documents are used in theportfolio, the contents list should highlight the relevant sections of these.

(2) A framework summary listing the programmes and free-standing modulesoffered by the school, including research-training, and giving quantitative dataon the human and physical resources available to the School for thisprovision, student numbers and projections, and data on retention, completionand progression to employment or further study, and, for post-experiencecourses, enhancement of careers.

It is recognised that the quantitative data available to Schools in some of these areasmay in parts be incomplete. The framework should contain or refer to such data asthe School uses in its own evaluations of the effectiveness of its programmes.

(3) An evaluation of the Strengths and Weaknesses of the School and itsacademic programmes, and the Opportunities and Threats facing it.

(4) A statement from the Dean of the Faculty (Optional)

The purpose of this item is to enable the Dean, if she or he wishes, to help set thecontext for the Internal Quality Audit in relation to the Faculty’s mission, to comment onthe School’s self evaluation, or to highlight particular matters for the attention of theaudit team. It is expected that this statement will have been made available to theSchool.

(5) A written submission, preferably a collective one, from students of theSchool, either in addition to or in place of a face-to-face meeting with studentsbefore the audit event. (Optional)

It is expected that the Head of School and the Director of Quality Assurance will drawthis opportunity to the attention of students and to facilitate the creation of arepresentative submission

(6) The School’s current statement of Learning and Teaching Strategy

(7) The reports from the two most recent Annual Monitoring Visits.


21

(8) For each of the undergraduate programmes, taught postgraduateprogrammes, and research training schemes within the scope of the audit:

(a) Programme specification and course regulations

(b) Most recent triennial course review report

(c) The most recent annual course review report (if this is not containedwithin the triennial course review report).

(d) Most recent two years’ reports from external examiners andSchools’ responses.

(e) Current student handbooks

(f) Where applicable, the most recent report from accreditation byProfessional or Statutory Bodies.

7.4 The Audit Secretary will advise the School of the number of copies of the SEPthat are to be provided.

7.5 Auditors will treat all documents provided by the School for the Audit, and alldiscussions and notes made at meetings, as confidential to the team and tothe School. Information derived from them will be used only for the purpose ofInternal Quality Audit.

7.6 The written student submission (if any) will usually be made available both tothe School and to the audit team. Exceptionally, the School’s studentacademic representatives or an officer of the Students Union may request thatthe written student submission be withheld from the School; in such a case,the Director of Quality Assurance and the Chair of the Quality AssuranceCommittee will adjudicate.


22

Section 8: The Internal Quality Audit Report and Summary

8.1 The Auditors will prepare a draft report based on the Audit, summarising theiranalyses and conclusions and recommendations under the following aspectheadings:


8.2 Each auditor will take responsibility for drafting an account of two of theaspects, including the evidence, analysis and judgement, and for identifyinggood practices or weaknesses that are to be highlighted in the reportsummary. Auditors should make use of the Aide Memoire (section 10)

8.3 The audit secretary will integrate the various contributions into the draft reportand will be responsible for the format of the report and uniformity of style.Auditors will comment on the first draft and the audit chair will sign off the draftreport as an accurate reflection of the findings of the audit team.

8.4 The Audit Report and Summary Report will be written in such a way as torespect the confidentiality of the written or oral contributions of any individualstaff member or student.

8.5 Within four weeks of the audit event, the audit secretary will send the draftreport and the summary for publication to the Head of School for correction oferrors of fact or of interpretation. The School will have two weeks in which tonotify the Quality Assurance Office of any errors of fact or of interpretation.

8.6 Within eight weeks of the audit event, the final version of the audit report andof the summary for publication will be sent to the Head of School, the Dean ofFaculty, and the Chair of Quality Assurance Committee.

8.7 The School will have a period of four weeks to develop an action plan inresponse to the Audit Report, to sustain strengths and remedy weaknesses.By the end of this period, the School must seek and gain the approval of theDean of the Faculty for this action plan and submit the action plan to theQuality Assurance Committee so that it can be considered by QualityAssurance Committee.

8.8 The Audit Secretary will check the action plan for completeness, raising anynecessary questions with the School and Dean concerned, before submissionto Quality Assurance Committee in the interests of efficient use of theCommittee’s time. As soon as possible thereafter, the audit report, action plan


23

and the summary for publication will be considered by Quality AssuranceCommittee.

8.9 Quality Assurance Committee may accept the findings of the internal qualityaudit in whole or in part and/or may require follow-up action by the Chair ofQuality Assurance Committee or by the Director of Quality Assurance. Follow-up actions might include a repeat audit or partial repeat audit orrecommendations to the Senate.

8.10 When Quality Assurance Committee has considered the audit report andaction plan, the report with the action plan as an annex, will be published onthe university’s intranet in order to foster enhancement through thedissemination of good practice. In exceptional circumstances, Heads ofSchool with the support of the Dean may seek exemption from this in whole orin part. Applications for exemption will be decided by Quality AssuranceCommittee or by its Chair or by the Director of Quality Assurance.

8.11 As part of its consideration of the audit report and action plan, QualityAssurance Committee will either authorise the summary report for publicationon the TQI website or withhold publication pending some action.


24

Section 9:Sample timetable and agenda for an internal quality audit event

External Adviser is invited to stay overnight before the Audit event as guest of theDirector of Quality Assurance. At discretion of Chair of QAC, External Advisor isinvited to dine with one or more of: a Pro-Vice-Chancellor, Dean of the Faculty whichincludes the School, Director of Quality Assurance (or Deputy), Chair of the AuditTeam.

8.30 a.m. Audit Team assembles

9.15 a.m. Meeting with Head of School

Present: Audit team and Head of School

9.30 a.m. Strategy, Planning & Resources

Present: Audit team, Head of School, Directors of Study, QualityAssurance Coordinator, and other relevant senior staff

10.15 a.m. Student Support:

Present: Audit team, Directors of Study, Admissions Officer, YearTutors, Learning Support Officers, Office Manager, and other relevantacademic and support staff.

11.15 a.m. Coffee and Private Team Meeting

11.30 a.m. Learning and Teaching:

Present: Audit team, Directors of Study, Examinations Officer, ITcoordinator, Quality Assurance Coordinator, representative academicand support staff.

12.30 p.m. Lunch and meeting with current and former undergraduatestudents, including student members of the SSLC

Present: Audit team and students.

Where appropriate to the programmes under audit and where feasible, auditors maywish to meet separately representatives of Public, Statutory or Regulatory Bodies(PSRB), commissioning agencies or employers

1.30 p.m. Private Team Meeting


25

1.45 p.m. Team to tour School facilities and External Adviser to tourUniversity Library and other relevant central services

Present: Audit team, relevant School liaison officers, and link personsfrom central services (e.g. Link Librarian)

2.45 p.m. Meeting with postgraduates involved in teaching and assessment

Present: Audit team and postgraduate students.

3.15 p.m. Private Team Meeting and Coffee/Tea

3.30 p.m. Concluding Meeting

Present: Audit team, Head of School, Directors of Study, QualityAssurance Coordinator, and other relevant senior staff

4.15 p.m. Private Team Meeting to discuss principal conclusions

5.00 p.m. Team disperses

The External Adviser is requested to provide a brief written report (report form to beprovided), on the day of the Audit or within one week, and this is made available tothe auditors together with summaries provided by the note taker to help them indrafting their sections of the audit report. Liaison over the draft report may be byface-to-face meeting but is more usually conducted by e-mail.


26

Section 10: Aspects of provision - an aide-memoire for Schools and auditors

(adapted from on Annex E of the Handbook for Academic Review © QAA 2004)

10.1 The aspects of provision for Internal Quality Audit are:


10.2 The following series of prompts is intended to assist Schools in evaluatingtheir programmes and arrangements and auditors in the analysis of theself-evaluation prior to the audit, collection of evidence during the audit, and preparation and compilation of the audit report.

10.2.1 Aims, intended learning outcomes and the curricula

Evaluation of the intended learning outcomes in relation to external reference points and to the broad aims of the programmes

• What are the intended learning outcomes for a programme and arethey stated clearly?

• How do they relate to external reference points including relevantsubject benchmark statements, the qualifications framework and anyprofessional body requirements?

• How do they relate to and are they appropriate to the overallprogramme aims?

The effectiveness with which the curricula are planned, designed andapproved to facilitate achievement of the intended outcomes

• How does the School ensure that curriculum content enables studentsto achieve the intended learning outcomes?

• How does the School ensure that the design and organisation of thecurriculum is effective in promoting student learning and achievementof the intended learning outcomes?

The means by which the intended outcomes are communicated to students,staff and external examiners


27

• How are the intended outcomes of a programme and its constituentparts communicated to staff, students and external examiners?

• Do the students know what is expected of them?

Evaluation of the means by which the School creates the conditions forachievement of the intended learning outcomes

• Do the design and content of the curricula encourage achievement ofthe intended learning outcomes in terms of knowledge andunderstanding, cognitive skills, subject specific skills (includingpractical/professional skills), transferable skills, progression toemployment and/or further study, and personal development?

• Are the design and content of the curriculum for each programmeappropriate to its potential for enabling students to achieve the intendedlearning outcomes.

• Is there evidence that curricular content and design is adequatelyinformed by recent developments in techniques of teaching andlearning, by current research and scholarship, and by any changes inrelevant occupational or professional requirements?

10.2.2 Teaching and learning

Evaluation of the quality of the learning opportunities offered by the School,the quality of the teaching delivered by staff, and the extent to which teachingand learning contributes to the achievement of the intended outcomes.

• How effective is the teaching of subject knowledge in relation tocurriculum content and programme aims?

• How effective is the teaching of subject-specific, transferable andpractical skills in relation to curriculum content and programme aims?

• Does the teaching have sufficient variety, breadth, depth, pace andchallenge?

• How effectively do staff draw upon their research, scholarship orprofessional activity to inform their teaching?

• How good are the materials provided to support learning? • Is there effective engagement with and participation by students? • Is the quality of teaching maintained and enhanced through effective

staff development, peer review of teaching, integration of part-time andvisiting staff, effective team teaching, and induction and mentoring ofnew, inexperienced or part-time staff?

• How effectively is learning facilitated in terms of student workloads?


28

10.2.3 Assessment

Evaluation of the assessment process and the standards of achievement ofintended programme outcomes that it demonstrates, and of the extent towhich student achievement meets appropriate expectations.

• Do the assessment process overall and the particular assessmentinstruments chosen enable learners to demonstrate achievement of theintended outcomes?

• Are there criteria that enable internal and external examiners todistinguish between different categories of achievement?

• Can there be full confidence in the security and integrity of assessment procedures?

• Does the assessment strategy have an adequate formative function indeveloping student abilities?

• Do the assessment processes comply with university policy andregulations on assessment and on external examining (regulations 8,8a and 9)?

• What evidence is there that the standards achieved by learners meetthe minimum expectations for the award, as measured against therelevant subject benchmarks and the qualifications framework (FHEQ)?

10.2.4 Student progression and student support

Evaluation of the School’s arrangements for recruitment and induction ofstudents and the strategy for supporting students, and their effectiveness infacilitating student progression towards successful completion of theirprogrammes.

• Is there an appropriate overall strategy for academic support, includinghandbooks and other written guidance, which is consistent with thestudent profile and the overall aims of the programmes?

• Are there effective arrangements for admission and induction which are generally understood by staff and applicants?

• How effectively is learning facilitated by academic guidance, formativeand summative feedback and supervisory arrangements?

• Are the School’s arrangements for academic and personal tutorialsupport consistent with university policy and clear and generallyunderstood by staff and students?


29

10.2.5 Learning resources

Evaluation of the effectiveness with which qualified staff are contributing to the achievement of the intended outcomes, and of the availability of appropriateSchool and central learning resources and the effectiveness of theirdeployment in support of the intended outcomes.

• Is the collective expertise of the academic staff suitable and availablefor effective delivery of the curricula, for the overall teaching, learningand assessment strategy, and for the achievement of the intendedlearning outcomes?

• Are appropriate staff development opportunities available?• Is appropriate technical and administrative support available? • Is there an overall strategy for the deployment of learning resources? • How effectively is learning facilitated in terms of the provision of

resources?• Is suitable teaching and learning accommodation available? • Are the subject book and periodical stocks appropriate and accessible?• Are suitable equipment and appropriate IT facilities available to

learners? • Are suitable equipment and appropriate IT facilities available to

teachers?

10.2.6 Enhancement of quality and standards

Evaluation of the School’s approaches to reviewing and improving the qualityand effectiveness of its teaching, and of the standards set and achieved by itsstudents.

• How does the School review and calibrate its standards and are theseprocesses effective?

• Does the School seek to enhance standards?• Is enhancement promoted effectively?


30

1. Are the academic standards of the educational programmes covered by theaudit appropriate to the qualifications awarded?

a) Undergraduate programmes YES/NO

b) Taught postgraduate programmes YES/NO

c) Research training for postgraduate research students YES/NO

Further comments:

Section 11: Report form for use by External Adviser

Academic Affairs

Quality Assurance and Standards Division

Internal Quality Audit: External Adviser’s Report

School:

External Adviser:

Date of audit event: Date of this report:

NotesExternal Advisers are invited to use this form to record their judgements andcomments, if possible, on the day of the audit. The completed form should begiven or sent to the Audit Secretary.

External Advisers may separately bring any serious concern about academicstandards to the attention of Professor JV Finch, Vice-Chancellor.

Continued overleaf


31

2. On the available evidence, the learning resources available to students are:

GOOD SATISFACTORY POOR

Further comments:

3. In the light of the School’s statements about educational aims and intendedlearning outcomes, please comment on any noteworthy features ofcurricular design in the undergraduate programme.

Continued overleaf


32

4. With reference to the programmes audited, are there major gaps in thecoverage of the subject?

YES/NO

Further comments:

5. Is there satisfactory provision for student support and guidance?

a) Undergraduate students YES/NO

b) Postgraduates on taught courses YES/NO

c) Research students YES/NO

Further comments:

Continued overleaf


33

6. Are there any aspects of student progression and achievement which yousee as worrying?

YES/NO

Further comments:

7. Are there any particular features in the educational practice of the Schoolwhich merit commendation?

YES/NO

Please specify:

Continued overleaf


34

8. Are there any matters of concern which the audit team and the Schoolshould consider?

Please specify:

9. Comment on Keele’s internal quality audit process will be welcomed. Please offer your advice in the space below.

Thank you for your assistance.

Professor R C J CocksPro Vice-Chancellor

Qao/cp/bm/k:mydocs/iqa/adviser/advisrep2006 15th December 2005

internal quality audit handbook - quality – safety · pdf filekeele university internal...

Documents