ECER 200719-22 September

Contested Qualities of Educational Research

Session 3a

Reflection and Self Evaluation for Teachers and Trainers

Gerald Heidegger and Irina Michel

How should Self Evaluation be approached?

National and international discussions concentrate too much on quality assurance

Development of quality is strongly

connected to self evaluation and reflection of teachers and trainers

Reflection gives a systematic possibility to dectect actions to improve quality

Aspects of the history of quality management and its tranfer to the educational sector

- The idea of quality management stems from production industry (ISO 9000)

- During the last decade the idea of quality management has been broadly transfered to non-profit organisations (EFQM – European Foundation for Quality Management)

- The first non-profit sector that has taken over quality management was the care sector – soon stressing self-evaluation (because a lack of „hard“ outcomes)

- Today a lot of public schools go through evaluation processes.

- For disadvantaged young people: Our method QSED (Quality through Self-Evaluation and Development)

Contexts of Evaluation

common understanding of the

phenomenon that should be evaluated

historical and cultural embeddedness of the

phenomenon of evaluation

national annd international evaluation discussions

Output indicators or

personal development

From ISO 9000 via EFQM to Q2E, QSED

different evaluation interests, fears and,

targets

Carried out by OECD, CEDEFOP

Evaluation trends

Carried out by IDEA, IOCE

Indicator centered evaluation

innovative evaluation approaches

targeting empowerment or

participation

strict, highly structured procedures Measuring soft

outcomes

External EvaluationAdvantages• Seemingly „objective

„ Reliable Outcomes• Easily transferable

results• No self-deception• No cheating (???)

• Should be combined with (internal) self-evaluation

• In order to….

Disadvantages• To measure „soft“

outcomes is very difficult/impossible

• Processes are aiming at the targets of the external evaluation

• In this way processes are severely narrowed down

• Teachers/Trainers are objects of external powers

• They become disempowered

• They become passive

• avoid these

staff orientation(3)

staff contentedness

(7)

processes(5)

societal responsibility and image (8)

resources(4)

customer contentedness

/satisfaction (6)

company results(9)

criteria of EFQM

politic and strategy

(2)

leadership(1)

Figure 6.1: Criteria of EFQM

Existing European evaluation approaches: example EFQM

… basic instrument for focus evaluation…as basis for the

development of a quality handbook

… as basis for a quality comparison between

schools

...basic instrument for the development of a

school specific evaluation instrument

…help for the development of a school specific profile

of strengths and weaknesses

… basic instrument for the development of a

quality model

Q2E can be used as ...

Figure 6.4: Options for using Q2E

How should Self Evaluation be approached?

Quality Development through Self Evaluation with a Web Based

Tool? CVET actors from six European countries were involved in the

construction process from the beginning of the project

„REFLECTIVE EVALUATION“

The main product: a web based tool is centred on self evaluation of

teachers/trainers & adaptable to specific cultural/ institutional needs

FOR

People at risk in the labour market

(at threat of being fired; unemployed; women returners)

AND

Participants in conventional CVT courses !

Quality Developmentthrough Self Evaluation with a Web Based Tool?

• Interactive web based tool• Six dimensions (reflection areas)• Four Levels• Adaption through the users themselves • Pictures, audio, video • Multiple methods for reflection • Material pool

Six dimensions

Reflective Evaluation

Self-Evaluation &

Quality Development

CCollaboration &

Networking

AAssessment &Recognition

Work Work ConditionsConditions

Values &Values & RolesRoles

Teaching & Teaching & LearningLearning

next

Four Levels

• System

• Institution

• Team/Interpersonal

• Individual

ChallengesSystem Level

- Criteria for QM dictated from outside

- Stiff quality assurance instead of dynamic qualitydevelopment

Institutional Level- Adaption to market oriented results instead of own aims

Team-/Individual Level- Resentment to participate actively

- Feeling forced by QM

Solutions

SystemAcknowledgement of „soft“ outcomes

InstitutionsExternal evaluation combined with internal self evaluation

Team/IndividualFor an easier and continuing adjustment to self evaluation for example our self reliant quality development tool for higher motivation through interactivity and multimedia