quality assessment criteria for language centres self...

Quality Assessment Criteria for Language Centres - Self-assessment Checklist

Quality Assessment Criteria for Language Centres

Self-assessment Checklist

Frascati, 25-27 November 2010



2

This document was produced at the Seminar “Guidelines for the Assessment of European Language Centres” held in Frascati, November 25-27, 2010.

Participants: Carmen Argondizzo University of Calabria Dominique Borel King's College London Maria Bronikowska Technical University of Lodz, Foreign Languages Centre Michelle Crick C.S.I.M. Universidad Complutense De Madrid Darja Dankova Brno University of Technology, Faculty of Civil Eng., Institute Of Social Sciences Paola Evangelisti University of Rome “Foro Italico” Alessandra Fazio University of Rome “Foro Italico” Johann Fischer University of Göttingen Michela Giordano University of Cagliari Maurizio Gotti University of Bergamo Heather Henshaw South East European University Peter Howarth University of Leeds Veronika Kareva South East European University Andrea Koblizkova University of Pardubice Jacqueline May Hochschule Der Medien (Stuttgart Media University) Michela Menghini University of Rome “Foro Italico” Christoph Nickenig Free University of Bolzano Salomi Papadima Cyprus University of Technology Ana Laura Rodriguez Redondo C.S.I.M Universidad Complutense De Madrid Danuta Soltyska Warsaw University of Technology Liliana Szczuka-Dorna Poznan University of Technology Christopher Taylor University of Trieste Jessica Ann Thonn University of Firenze Ruth Tobias Hochschule Darmstadt/Sprachenzentrum Irmgard Wanner University of Leipzig



3

Quality Assessment Criteria for Language Centres Self-assessment checklist

Frascati, November 2010

1. This document is intended for the LC’s use in order to audit its services and procedures, and forms the basis of the Summary Self-evaluation Report. A structure of the Summary Report is provided in the ‘LC External Review’ guidelines, under the Self-Evaluation Report section.

2. ‘Possible Evidence’ is a list of suggested evidence the Centre should provide in order to justify its point. Where there is no evidence listed, the LC should provide a written statement addressing the topic.

3. For each question indicate on the scale either the level of satisfaction (Sat) with the topic or the level of agreement (Agr) with the statement. In addition, you are asked to indicate the level of importance (Imp) of this topic to the LC. If the LC does not have the document/service/topic the question is asking about, circle “Not Applicable” (N/A).

Scale of Importance (Imp) Scale of Satisfaction (Sat)

Scale of Agreement (Agr)

(1) not important to the LC (1) very dissatisfied (1) strongly disagree (2) of little importance to the LC

(2) dissatisfied (2) disagree

(3) moderately important to the LC

(3) could be improved (3) undecided

(4) very important to the LC (4) satisfied (4) agree

(5) of utmost importance to the LC

(5) very satisfied (5) strongly agree



4

Quality Assessment Criteria for Language Centres Self-assessment checklist

Frascati, November 2010

Possible Evidence Self-assessment scale

STATEMENT OF PRINCIPLE

1. Mission / Vision: general statement Type

a. Does the Language Centre (LC) have a mission statement with a clear vision?

Mission statement LC statute

Agr 1 2 3 4 5 N/A

b. Are the university mission and the LC mission compatible?

LC mission University mission

Agr 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

c. Is the university language policy clearly defined?

University Language Policy Other related policies

Agr 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

d. How involved is/was the LC in defining it? Written statement Sat 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

2. Needs identification and strategic planning

a. How strong is the needs analysis which is carried out at the Centre?

Written statement Sat 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

b. How strong are the strategic planning processes of the LC?


Imp 1 2 3 4 5 N/A

c. Is there a short-term or long-term plan of allocation of resources (expenses)?

Written statement Agr 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

3. Aims and more specific objectives

a. How clearly defined are the LC's aims?

LC statute Sat 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

b. Is internationalization one of the aims?

Foreign student intake Teacher and administrative staff mobility International research programs Agreements/contracts with foreign universities

Agr 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

c. How actively does the LC support mobility?

Staff exchange programs Student exchange programs European Credit Transfer System (ECTS)

Sat 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

ORGANISATIONAL STRUCTURE

1. Stakeholders Type



5

a. Has the LC identified fully its stakeholders? 1

LC Statute LC website LC student handbook

Sat 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

b. How well does the LC involve stakeholders? 2

written statement Sat 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

c. How well does the LC communicate with their stakeholders?

List of communication forms Calendar of communications

Sat 1 2 3 4 5 N/A

Imp 1 2 3 4 5 N/A

2. Structure

a. Does the organigram clearly show roles and lines of responsibility?

Organigram Sat 1 2 3 4 5 N/A

b. Does the LC comply with legal requirements?

LC Rules and Regulations Certificates of compliance for structural soundness, occupational health and safety, etc

Agr 1 2 3 4 5 N/A

c. Does the LC comply with university statutes?

University statute Agr 1 2 3 4 5 N/A

d. How effective is the LC’s position within the university structure?


e. How is the LC’s structure fit for purpose?

LC structure + organigramme

Sat 1 2 3 4 5 N/A

f. How clear are the lines of responsibility?

LC website, handbooks job descriptions

Sat 1 2 3 4 5 N/A

g. On which levels is the LC involved in university decision-making?3

Open-ended Sat N/A

h. How effective is the leadership and management of the centre in supporting students and staff?

Instrument Sat 1 2 3 4 5 N/A

3. Planning and funding

a. Is the LC part of the strategic planning processes?

University statute LC statute

Sat 1 2 3 4 5 N/A

b. To what degree does the LC have effective planning processes in place?


c. How far is the LC involved in setting its own strategic planning targets?


Imp 1 2 3 4 5 N/A

d. What kind of financial resources does the LC get and are they appropriately

Budget; open-ended Sat N/A

1 Stakeholders may be: students, students’ parents, teachers, LC board, university governing bodies, faculties, university

services (e.g. computer centre, resource centres), national and local government authorities (e.g. higher education ministry, political administration), schools, external bodies and outside clients, local economy, future employers, foreign universities, professional networks and associations. 2 E.g. Representation, Evaluation, Involvement in program design

3 E.g., in academic bodies, administrative bodies



6

allocated?

e. Does the allocation of resources adequately reflect the LC’s mission and strategic planning?


4. Orientation and support of staff4

a. Is the staff well informed, updated, sufficient, available?

Sat 1 2 3 4 5 N/A

b. Is the information (on- and offline) up-to-date, helpful and sufficient?

LC website LC student handbook List of communication forms

Sat 1 2 3 4 5 N/A

c. What kind of guidance and orientation are provided for new staff (teachers, admin, technical)?

LC statute orientation documents

Sat 1 2 3 4 5 N/A

d. What staff feedback on orientation and support is there?


e. How effective is LC support for staff? Sat 1 2 3 4 5 N/A

HUMAN RESOURCES 1. Personnel Type

a. Does the staff5’s employment status affect the LC's performance?

written statement

Agr 1 2 3 4 5 N/A

b. How adequate are the LC human resources, human resource policy and practice to current and future needs (e.g., gender policy, age profile, recruitment, promotion, redeployment and staff development)?

LC statute/handbook Employment contracts

Sat 1 2 3 4 5 N/A

c. What are the terms of recruitment, employment and retention for the following groups? Are they effective?

For each of the following categories provide:

Job description Employment contracts Recruitment procedures Turnover rates

i. Teaching staff (permanent, freelance, interns);

Agr 1 2 3 4 5 N/A

ii. Administrative staff and Technicians (full-time, part-time, specialists, volunteer, interns, student workers)

Agr 1 2 3 4 5 N/A

iii. Volunteers, interns, work-study students; tutors, peer tutors

Agr 1 2 3 4 5 N/A

iv. Porters, cleaning services, etc. Agr 1 2 3 4 5 N/A

d. Do the staff’s qualifications meet the LC’s needs?

CVs written statement

Agr 1 2 3 4 5 N/A

2. Management tools

a. Are the role and the status of the leadership effective?

written statement Assessment by all staff members

Agr 1 2 3 4 5 N/A

4 Staff, in this instance, refers to teachers, administrative personnel and technical personnel 5 In this case, ‘staff’ refers to all the categories described in 1.c i, ii, iii and iv, unless otherwise specified.



7

b. Is management appropriately supported and trained?

Certificates of completion of training courses

Agr 1 2 3 4 5 N/A

c. Are regular feedback and appraisal provided to all staff on their performance?

Appraisal forms Appraisal calendar

Agr 1 2 3 4 5 N/A

d. How positive is the organizational climate?

Organizational Climate Survey

Sat 1 2 3 4 5 N/A

e. What is the degree of participation, involvement and commitment of the staff?

Number of teacher-staff meetings, number of teacher-teacher meetings, attendance/absenteeism rate in meetings

Sat 1 2 3 4 5 N/A

f. How effective is the management of staff performance?

Reward and reprimand system; assessment carried out by all staff members; keeping track of staff hours, teacher registers, management procedures

Sat 1 2 3 4 5 N/A

g. How clearly are processes and workflows defined?

Organigram Job descriptions

Sat 1 2 3 4 5 N/A

h. How well do official descriptions and actual tasks performed match?

self-assessment carried out by all staff members

Sat 1 2 3 4 5 N/A

3. Professional standards and quality mechanisms

a. Do the professional standards match the requirements of the LC?

Agr 1 2 3 4 5 N/A

b. Is there a university and/or a LC quality policy?

University quality policy LC quality statement

Sat 1 2 3 4 5 N/A

c. Are there effective quality assurance mechanisms in place?

Agr 1 2 3 4 5 N/A

4. Communication

a. Does the LC communicate efficiently with its internal stakeholders?

Forms of communication (on- and off-line)

Agr 1 2 3 4 5 N/A

b. Does the LC communicate efficiently with external stakeholders?

Forms of communication (on- and off-line) Advertising

Agr 1 2 3 4 5 N/A

c. Does the LC communicate efficiently with its staff?

Assessment by staff Agr 1 2 3 4 5 N/A

FACILITIES

1. Facilities for staff and students: Type

a. Does the LC offer suitable and comfortable premises?

Offices, teachers’ rooms, , Self-Access Centre6

Agr 1 2 3 4 5 N/A

b. As for classrooms, is the number of classrooms: number of Sat 1 2 3 4 5 N/A

6 availability; accessibility; convenience of locations, conformity to legal requirements; maintenance; portering; ownership -

exclusive vs. shared usage; cleanliness, furniture, lighting, heating, acoustics).



8

seats adequate to the number of users?

seats / of users

c. As for self-study labs, is the ratio between the number of seats and hours of use adequate to the total number of users?

Self-study labs: number of seats / of hours / of users

Sat 1 2 3 4 5 N/A

d. How effective are the selection and purchasing procedures for furniture and equipment?

Acquisition policies Selection procedures

Sat 1 2 3 4 5 N/A

2. Technological infrastructure

a. Is the teaching and learning equipment up-to-date?

Agr 1 2 3 4 5 N/A

b. Are learning resources available and accessible for all students?

Accessibility policies Agr 1 2 3 4 5 N/A

c. What are the updating procedures & policies?

Upgrading and updating procedures & policies

Sat 1 2 3 4 5 N/A

d. Is the Virtual Learning Environment7 functional and user-friendly?

Staff and Student opinion survey

Sat 1 2 3 4 5 N/A

e. Is feedback and input from teachers regularly used to make IT decisions?

Agr 1 2 3 4 5 N/A

TEACHING, LEARNING AND ASSESSMENT 1. Management of teaching and courses

a. Do the students have easy access to the information about the courses?

Forms of communication (on- and off-line)

Agr 1 2 3 4 5 N/A

b. How efficient are the enrolment and registering procedures?

Opinion survey by teachers, staff, management and students

Sat 1 2 3 4 5 N/A

c. How efficient is the course organisation and scheduling?

Opinion survey by teachers, staff, and management

Sat 1 2 3 4 5 N/A

d. Is scheduling independent of the official university course scheduling?

Agr 1 2 3 4 5 N/A

e. How efficiently are the students streamed into the courses?

Opinion survey by teachers Student assessment; placement tests

Sat 1 2 3 4 5 N/A

f. How far do the courses meet the needs and objectives required by the university degree programmes?

Sat 1 2 3 4 5 N/A

g. How far do the courses correspond to the European Framework?

Sat 1 2 3 4 5 N/A

h. Is feedback from teachers regularly used to make programming decisions?

Agr 1 2 3 4 5 N/A

7 E.g. IT equipment, server, technical support,



9

2. Teaching

a. Do course syllabuses have clearly defined learning outcomes?

Course descriptions Agr 1 2 3 4 5 N/A

b. How far do learning outcomes meet the needs of the students?

Sat 1 2 3 4 5 N/A

c. Is the methodology adopted coherent with syllabuses and evaluation tools?

Teacher self-assessment tool

Agr 1 2 3 4 5 N/A

d. Are appropriate materials accessible for teachers?

Teacher opinion survey Agr 1 2 3 4 5 N/A

e. What forms of learning support and advice are available to students?8

Lists of forms Schedules

Sat 1 2 3 4 5 N/A

f. Does the LC provide effective additional support to students with special needs?

Agr 1 2 3 4 5 N/A

Sat 1 2 3 4 5 N/A

g. How effective are material selection and purchasing procedures?

Acquisition policies, selection procedures, material development Staff opinion survey

Sat 1 2 3 4 5 N/A

h. Is feedback from teachers regularly used to make material acquisition decisions?

Agr 1 2 3 4 5 N/A

3. Assessment

a. What forms of evaluation/assessment are adopted?9

List of assessment forms offered by LC; open-ended q.

Sat N/A

b. Are exams organised by external organisations?

List of external forms of assessment

Agr 1 2 3 4 5 N/A

c. What format are the LC’s tests?10 List of assessment types Sat 1 2 3 4 5 N/A

d. On what criteria are they marked?11 Marking criteria, data collection and evaluation of results; open-ended

Sat N/A

e. Are the reliability and validity of the tests checked?

Agr 1 2 3 4 5 N/A

f. Are criteria of transparency respected?

Feedback procedures to students Feedback procedures from students Accessibility of reports

Agr 1 2 3 4 5 N/A

g. Does the LC provide special tests for students with special needs?

Written statement Agr 1 2 3 4 5 N/A

h. How satisfied are you with the assessment methodologies used in the

Sat 1 2 3 4 5 N/A

8 E.g. Tutoring, teacher consultation, self-study programs, face-to-face programme, tandem programs

9 E.g. Placement tests, self-assessment, ELPs, continuous assessment, formative/summative assessment, level exams,

proficiency tests, certifications 10

E.g. paper or computer-based, types of skills assessed, types of tasks required 11

E.g., objectivity, authenticity, relevance



10

LC?

i. How satisfied are you with the overall results of your students?

Sat 1 2 3 4 5 N/A

j. How satisfied are you with the retention of your students?

Sat 1 2 3 4 5 N/A

k. Do all groups of students (e.g. gender, ethnicity) achieve equally and to their potential?

Agr 1 2 3 4 5 N/A

4. Students’ evaluation of programmes and services

a. Does the LC offer the opportunity of an end-of-course evaluation by means of questionnaire and/or feedback?

Agr 1 2 3 4 5 N/A

b. Are results made available to users and staff?

Agr 1 2 3 4 5 N/A

c. Is feedback from users taken into account in order to enhance services and facilities?

Agr 1 2 3 4 5 N/A

OTHER LANGUAGE-RELATED ACTIVITIES

1. Does the LC offer other activities such as: Type

a. Courses for mobility for students and staff?

Agr 1 2 3 4 5 N/A

b. Courses for university administrative personnel?

Agr 1 2 3 4 5 N/A

c. Courses addressed to users outside the university context12?

Agr 1 2 3 4 5 N/A

d. Tandem? Agr 1 2 3 4 5 N/A

e. Language advising? Agr 1 2 3 4 5 N/A

f. Training seminars13? Agr 1 2 3 4 5 N/A

g. Translation and Interpreting? Agr 1 2 3 4 5 N/A

h. Testing services? Agr 1 2 3 4 5 N/A

i. A Newsletter? Agr 1 2 3 4 5 N/A

WELFARE AND STUDENT SERVICES

1. Does the LC provide accommodations?

Agr 1 2 3 4 5 N/A

2. On what basis / condition is it provided?

Open-ended

N/A

3. Does the LC provide catering? If so, how?

Agr 1 2 3 4 5 N/A

4. Are there social programmes and extracurricular activities?

Agr 1 2 3 4 5 N/A

5. Does the LC provide information Student handbook Agr 1 2 3 4 5 N/A

12E.g., general public, school teachers, company staff, army, etc. 13E.g. Teacher training (to non-LC teachers), technical training, intercultural communication, Moodle, etc.



11

about grants? LC website

RESEARCH

1. What type of research does the LC carry out?

a. Teaching-Led Research Research requests/ statements

Agr 1 2 3 4 5 N/A

b. Learning-Led Research Research requests/ statements

Agr 1 2 3 4 5 N/A

2. What is the impact of research on the LC activities?

Sat 1 2 3 4 5 N/A

3. How many research projects are currently being carried out in the LC?

Sat 1 2 3 4 5 N/A

4. On which scale is research carried out?14

Sat 1 2 3 4 5 N/A

5. Where does the funding come from?

List of funding means / bodies

Sat 1 2 3 4 5 N/A

6. How many teaching & learning projects are carried out? (EU, national, regional, university-internal, etc.)

Sat 1 2 3 4 5 N/A

7. Are research results effectively communicated to LC staff members?

Agr 1 2 3 4 5 N/A

Sat 1 2 3 4 5 N/A

RESULTS ANALYSIS + IMPROVEMENT

1. Do the authorities of the Faculty/University ask for the results of the analysis and the measures taken for improvement?

Do the Faculty/University authorities enquire about the results of this QA procedure and the measures taken for improvement?

Agr 1 2 3 4 5 N/A N/A

2. To what extent are the results taken into account for improvement?

a. How have the results of the data gathering and evaluation results impacted the activities?

b. How is the link between these results and institutional planning and development processes ensured?

Agr 1 2 3 4 5 N/A

Sat 1 2 3 4 5 N/A

Sat 1 2 3 4 5 N/A

3. To what extent does improvement Imp 1 2 3 4 5 N/A

14

Individual, collaborative within the LC and with other university departments, with other university LCs, national and/or international?



12

depend on external factors? Sat 1 2 3 4 5 N/A

4. How will the results of this QA assessment be communicated to stakeholders?

List of communication forms

quality assessment criteria for language centres self...

Documents