galya mateva, laura muresan - internal assessment: an instrument for quality management in language...

INTERNAL ASSESSMENT: AN INSTRUMENT FOR QUALITY MANAGEMENT IN LANGUAGE INSTITUTIONS Galya Mateva, OPTIMA, BulgariaLaura Muresan, QUEST Romania

2

LLL-Project:Improving Standards of Quality

in Adult Language Learning

(Grundtvig Learning Partnership)

3

ACKNOWLEDGEMENTSEXPERIENCE AND INSPIRATION DRAWN FROM : EAQUALS and the European language policy The National Associations for Quality Language

Services in Bulgaria, Romania, Greece and Croatia

The English and German Department of the University of Zaragoza

Research in the Area Synergies with previous European projects, e.g.

the ECML projects on “Quality Management” & “QualiTraining”, the Grundtvig project “IPALE”

Colleagues & network partners

4

PROJECT BACKGROUND

THE NEEDS OF THE FOLLOWINGBENEFICIARIES HAVE BEEN EXPLOREDAND TAKEN INTO CONSIDERATION:

ADULTS IN LANGUAGE EDUCATION IN THE PRIVATE SECTOR

UNIVERSITY LANGUAGE DEPARTMENTS BOTH WELL-ESTABLISHED & RECENTLY

FOUNDED ASSOCIATIONS OF QUALITY LANGUAGE SCHOOLS

5

AIMS OF THE PROJECT To create a well-balanced Quality

Improvement Scheme for internal assessment in the FOUR basic areas of MANAGEMENT, ACADEMIC SUPPORT TO TEACHING, TEACHING PROCESSES and LEARNING PROCESSES

To pilot the effectiveness of the Scheme as a management instrument in all partner institutions

To integrate internal and external assessment where applicable for the purpose of ensuring on-going quality assurance combined with professional consultancy

6

AIMS OF THE WORKSHOP To present the project product: The Quality

Improvement Scheme (v.4) To discuss aspects of the Scheme as an

active management instrument, including examples from piloting

To consider the relevance of the Scheme in the context of EAQUALS and individual schools

To enrich the Scheme with new ideas and perspectives

7

THE STRUCTURE OF THE QUALITY IMPROVEMENT SCHEME

QUALITY AREA

POSSIBLE QUESTIONS/ STATEMENTS FOR (SELF)ASSESSMENT

ASSESSMENT SCALE

EVIDENCE

8

9

MANAGEMENT

Human Resources Management

Management of Information and Communication Processes

Management of Services and the Environment

10

MANAGEMENT QUALITY AREA POSSIBLE QUESTIONS

М.15. Implementing innovative practices and services, identifying unique selling points

Is there a system for implementing innovative practices and services in your institution?

Do the innovative practices/services bring added value to your work?

Are your unique selling points easily identifiable?

11

ACADEMIC SUPPORT TO TEACHING QUALITY AREA POSSIBLE QUESTIONS

A.6. Effective observation systems and peer-cooperation framework

Is there an observation system in place? What types of observation are being used? Are all teachers observed at least once every academic year?Is observation documented and followed by feedback?Is there evidence of action taken on the basis of observation outcomes?Is there a framework for facilitating/encouraging peer-cooperation?If yes, what forms does collaborative work take?

12

TEACHING PROCESSESQUALITY AREA POSSIBLE QUESTIONS T.7. Awareness of learner profiles and learning styles

Are teachers aware of the learning styles and preferences of their students?Do they investigate them?Do teachers demonstrate individual approach to learners in keeping with their learning styles?

13

LEARNING QUALITY AREA POSSIBLE QUESTIONS L.5 Learners` self-analysis of learning

Do learners self-assess their language/communicative performance regularly?Can they correct their own mistakes?Are they in the habit of correcting each other?Do they use the CEFR can-do statements for self-analysis?Do they keep an up-to-date portfolio?Do they keep a dossier with their written work? Do they find it useful?

14

EXAMPLES & ACTIVITIES INVITATING FEEDBACK (1)

Management

15

PILOTING PHASES & EXAMPLES

PILOTING – EXAMPLES FROM SPAIN: THE EIIC TEACHERS’ NETWORK

Feedback on possible amendments/changes & improvements to draft the final version of the questionnaire

O The Greek teamO The Croatian TeamO Teachers from the Master in English StudiesO Teachers from the 1st year in the Degree of English

StudiesO Teachers from a UZ degree other than English StudiesO Teachers from a high school (public school)O Teachers from a high school (private school)O Teachers from a bilingual schoolO Teachers from a vocational schoolO Teachers from the Official School of Languages

PILOTING WITH LEARNERS

Target population (a stratified sample)

O 10 PhD students in EnglishO 10 postgraduate students at a master level (UZ Master in

English Studies)O 10 undergraduate students of the 1st year in the UZ Degree of

English StudiesO 10 undergraduate students in a UZ degree other than English

Studies - ESPO 10 students from the Official School of Languages O 10 young learners in higher education(public school)O 10 young learners in higher education(private school)O 10 young learners in higher education (private bilingual school)O 10 adult learners from a vocational school

9 Lifelong learning skills are promoted in my language class.

10 The teacher interacts well with the class.

11 The teacher shows me techniques to improve my language learning.

12 I know how my learning will be assessed / evaluated.

2 11 46 1

5 55

2 12 46

10 49 1

8 I find the class interesting and motivating.2 15 43

Methodology & pedagogical approach

Results of the piloting: example from piloting with learners in Spain

Don’t agreePartly agreeFully agree Don’t knowQuit

35 I learn the language by communicating with others (e.g. peers / students and teacher). 

36 In learning the language, I have opportunities to think and give my opinion about topics, ideas, etc.

37 In learning the language, I need to search for, understand and transmit information.

Professional skills

38 In learning the language I plan and organize my work on a regular basis to successfully acquire competence in the language. 

2 18 39 1

4 55 1

1 22 34 2 1

7 28 23 1 1

Don’t agreePartly agreeFully agree Don’t knowQuit

Results – sample (cont.)

Feedback re the section on Teaching Teachers found it thought-provoking, challenging

and a useful tool in self-assessing their teaching and the school’s academic coordination.

Novice teachers found it intriguing and they needed help from senior teachers/academic coordinators/mentors to elaborate on certain key areas. For example, blended learning - Core Inventory.

It focuses on the teachers’ abilities, the quality aspects of teaching, the academic coordinator’s role & a learner-centered approach on teaching. As a self-assessment tool it helps teachers reflect on their teaching.

Results of the piloting: example from piloting with teachers in Greece

“A scoring system would be appropriate, which would distinguish between core or primary features/ aspects to meet minimal standards and periphery or secondary features/ aspects needed in order to assess level of mastery.”

(A.T., Manager – example from Romania)

DO YOU FIND THE SCHEME USEFUL? COULD IT BE A USEFUL SELF-ASSESSMENT INSTRUMENT IN YOUR ORGANIZATION? PLEASE EXPLAIN!

Results of the piloting: example from piloting with managers /academic

directors in Bulgaria, Croatia, Romania

22

EXAMPLES & ACTIVITIES INVITATING FEEDBACK (2)Academic Coordination of

Teaching

Process of Teaching

The Process of Learning

23

THE INTERNAL ASSESSMENT PROCEDURE: A MANAGEMENT TOOL Multiple perspective: Manager assessment Teacher self-assessment /assessment Teacher peer-assessment Learner self-assessment/assessment Learner evaluation of teaching and learning

opportunities

Overall picture of Internal assessment Matching Internal with External assessment Professional advice and growth

24

Thank you!