Issues in QM in language education

EAQUALS Workshop November 2013Frank Heyworth

The background• More than 20 years of development

of QM / QA procedures and practice by EAQUALS and others has led to:– A consensus on the importance of QM in

language education– Established and recognised practice of

inspections and audits

Good practice• Aspects of good practice which promote

commitment to quality, to innovation and continuous improvement, to enlightened leadership and teamwork; – regular observation of classes with peer

observation as a feature of self-assessment, – the institutionalization of reflective practice and

commitment to continuous professional development.

– basing action on fact gathering with regard to practice and reflective analysis of what is done

Some issues• A study of the «state of the art» of QM

raised questions:– Is practice backed by valid principles?– Is there a good fit between the aims of QM and the

methodology we use to implement and assess it?– How can we give real added value to repeat inspections?– Eliminating defects v. positive encouragement of quality– Is there proper research into its applications and effects?– Does it work?

Some Objections• (a) it doesn’t work – practically all institutions

are accepted in the long run; • (b) the cost-benefit ratio is unsatisfactory – it

takes time and resources which could more profitably be devoted to research and teaching;

• (c) it encourages institutions to be bureaucratic – the need for objective criteria causes them to concentrate on measurable and observable details, rather than the important ones;

• (d) the spread of quality assurance and pressure to comply to it is turning QA itself into a non-productive business. (Weber 2007)

Some more objections• “if standards are based on specific measurable

competences rather than a more holistic inter-personal view of communication and culture, the potential educative richness of language education may be lost. (Böttcher 2008)”

• “the price to be paid for injecting market pressure into secondary school education, for turning foreign language classrooms into arenas of competition for the best test results, for coating instruction with more and more layers of assessment, for reducing educational “quality” to a limited number of measurable performance indicators, and for conceiving of output or outcome as the linchpin of quality development, may be hefty and unacceptable.” (Kurtz 2011)

Models of quality 1The EAQUALS QM procedures are based implicitly on:•Viewing quality as a feature of the relationship with the client, whose satisfaction is the measure of the quality achieved, and ‘exceeding clients’ expectations’ as the goal – the charters are promises made to clients.

•Detailed management of processes in the three phases of design, implementation and outcome, with the aim of achieving ‘zero defect’ (i.e. fault-free operations) through inspection and control.

•zero defects – everything in the charters must be achieved: identified defects lead to “requirements”.

Models of quality 2• Cheng & Tam (1997) cite seven

models of quality in education: – the goals and specifications model;– the resources input model; – the process model; – the satisfaction model; – the legitimacy model; – the absence of problems model; – and the organizational learning model

Models of quality 3• Fitness of purpose

– Doing the right things• Fitness for purpose

– Doing things right

Can all these models be valid?

• There are good (and less good) arguments for all of them, but it’s unsatisfactory to bundle so many different concepts into the same term «quality»

Client satisfaction• There are problems in applying notions of client

satisfaction to education. It is – an ‘experience’ good, one whose quality can only

be assessed after it has been experienced. – a ‘credence’ good – one whose long-term value

cannot be immediately identified even after it has been experienced.

– In most educational settings there is relatively little choice of institution for learners (except in the case of language schools and even here courses are paid for in advance).

– a fundamental mismatch between the concept of students as customers, requiring short-term satisfaction, and the long-term aims of education.

Fitness of and for purpose

Quality, process and procedures

• Process approaches are effective for eliminating defects, less so for generating creativity, originality..– Tends to be topdown– Applying systems can be rigid– They don’t always reflect the complexity

of learning and teaching, human differences…

A new paradigm?• Quality based on a combination of

trust and legitimacy– Procedures and practices which

contribute to establishing trust between stakeholders

– Systems, tools, resources which provide evidence to legitimate the trust

Trust & Legitimacy• ‘normative legitimacy’ - an activity generates

trust because it conforms to an accepted set of standards

• ‘empirical legitimacy’ - trust is based on people’s belief that implicit standards underlie practice.

Aloyo 2011

A hierarchy of quality• Eliminating low

quality– Procedures (complaints,

grievances etc.) which identify problems

– Systems for client feedback– Clear and accurate

information– Compliance with legal

requirements– Quality checks on teaching– Defined curricula and

syllabi

• Promoting high quality– Investment in training

and development– Reflection, research and

innovation– Developmental

observations– Negotiation of content– Involvement of all

stakeholders

Measures which build trust• To the outside the institution

– Charters and guarantees– Clear, honest communication– Transparent use of resources– Reliable, valid assessment and certification

• Within the institution– Commitment to training– Distributed leadership– Clear top-down and bottom-up communication– Shared reflective practices– Regular data collection – as a basis for action– Involvement of all with room for initiative and

creativity

Evidence that establishes legitimacy

• To the outside– Descriptive frameworks (CEFR, EPG, TDFRAM etc.) which

make practice cohereent and transparent– Availability of accurate information about aims, curriculum,

programmes, results– Regular external auditing with findings available to

stakeholders– Consistent record of fair dealing

• Inside the institution– Democratic practice with learners, teachers, administrators– Constructive meeting habits, clearly documented– Clarity about consultation and decision procedures– Developmental approaches to staff appraisals

Creating a culture of quality

A quality spiralvalid objectives

fitting concept

‘true’ implementation

honest assessmentKohler 2007

Issues for EAQUALS• Unpicking what is assessed

– Identification of defects?– Processes which develop trust?– Results of measures taken?– Evidence of impact?

• What do indicators indicate?– Avoiding lamination, wet paint syndrome– Recognising what is behind the indicators– Description and interpretation rather than

checklists

Areas for audits / inspections

• Should the inspection continue to be “requirements” led?

• How should first and subsequent inspections differ?

• Can we develop descriptive frameworks which stress the positive aspects of quality…. P

4 of DEMING’S 14 points3 Cease dependence on inspections5. Continually seek out problems10. Eliminate exhortations12. Institute education