12. assessment and evaluation · 2015-10-22 · presentation and practice ... chapter 12:...

Teaching English12. Assessment and evaluation

Nancy Grimm – Michael Meyer – Laurenz Volkmann

0. Table of contents

1. Achievement as social parameter

2. Functions of assessment and evaluation

3. Assessment and testing

4. What are we testing and how?

5. Grading

6. Correcting mistakes

7. Practical examples

8. Recommended reading

Chapter 12: Assessment and evaluation 2

Discuss:

Write down functions of tests at school suggested by the cartoon.

Add other possible functions of assessment and evaluation.

3Chapter 12: Assessment and evaluation

Terminology

Assessment: providing information about how teaching and learning processes can be diagnosed and improved

Evaluation: measures learning results and how they can be graded


Testing

(Leistungs-überprüfung)

Evaluation

(Leistungs-beurteilung)

Assessment

(Leistungs-messung)

cf. Finkbeiner 2012: 386


Contradictory concepts of achievement and performance

5

Performance principle Pedagogical

performance principle



Social functions of the performance principle (Leistungsprinzip)

6

Ensure distribution of rewards (esp. monetary gain and social status)

Foster productivity and prosperity

Regulate distribution of professional and social positions in society



Consider and discuss arguments why the competition-oriented performance principle is only partly suited to serve as the sole basis for evaluation at school.


In reality, coveted positions in our society are not at all solely bequeathed according to the performance an individual exhibits. Instead, the performance principle is only one among the principles of distribution of life chances [such as being liked or having the right social connections].

The principle of performance as give and take and its direct linkage with competition-driven relations between individuals is negative; it hampers and obstructs solidarity. Indeed, it supports processes of social fragmentation and the destructionof solidarity (Ent-solidarisierung).

The performance society equals stress, rivalry, and the struggle for self-identity.

Jürgens 2000: 16, 18, our trans.


Pedagogical performance concept (pädagogischer Leistungsbegriff)

8

At the center of a pedagogical concept of performance and assessment is the right of the student for individual care and support, being considered as a whole person and with regard to his or her learning as part of a group or community.

Jürgens & Sacher 2008: 28, our trans.



Pedagogical performance concept: 5 principles (concepts)

9

1. Achievement and assessment are “constructs” (Bohl 2009: 6)

2. Is influenced by nature and nurture

3. Is product- and process-oriented

4. Can be the result of individual or cooperative achievement

5. Should be interpreted in a holistic manner

Performance


2. Functions of assessment and

evaluation


Functions

Diagnosis

Information

Differentiation

Education

2. Functions of assessment and

evaluation

Reflect on memorable tests of your school career. Which of the functions did they serve?

What will be the functions of the tests you will have to pass yet?


Functions

Diagnosis

Informa-tion

Differentia-tion

Education

3.1 Parameters and criteria

Reference frames


Norm referencing (sachliche / kriteriale Bezugsnorm)

Group referencing (soziale Bezugsnorm)

Individual or self-referencing (individuelle Bezugsnorm)

13

Consider the following assessment problems / challenges:

1. Why do some class tests have (overall) extremely bad results, and others extremely good results?

2. Why does an average student get better grades in one class than in another?3. Why do some students feel that they can never get a good grade in a particular

class?4. If a student makes progress in a dictation test and has ‘only’ 35 mistakes

instead of his usual 45-50, should he still get a ‘no pass/fail’ (grade 6) or possibly – according to the concept of individual referencing – a better grade? How can this be justified vis-à-vis his or her classmates?



The ‘big three’

14

Validity (Gültigkeit)

Reliability (Zuverlässigkeit)

Objectivity (Objektivität)



Objectivity as most important and strictest criterion suggests assessment independent of subjective factors.

15

Objective test / task

types

True / false tasks

Multiple-choice tasks

Matching tasks

Construction tasks

Completion tasks



Since the yardstick of communicative competence is the ‘real world out there‘, discuss which competences cannot be tested with regard to the ‘big three’. Also, what other factors might come into play when teachers grade student performances?


3.2 Subjective factors

To err is human!


Feasibility: time factor

Interrelation of teaching and test results

Personality and culture of students

Influence of prior knowledge about the student on the teacher

Test formats and testing situation

Effects of grading

Extenuating circumstances

3.2 Subjective factors

The testing dilemma

18

� Defines the content (what is tested) and method (how it is tested)

� Administers the test

� Assesses and evaluates the results

� Corrects or adjusts test results according to his / her criteria

� Assesses the overall and individual outcome of the test

The individual teacher


3.3 Toward a fair assessment culture

Reflect on the guidelines for good and successful testing. Rank them according to their feasibility in school contexts considering for example an oral vocabulary test in comparison to a complex written test (Klassenarbeit).


Becoming systematic

Self-reflexivity

Non-judgmental observation

Focus on elements of performance

Avoiding stereotyping

‘Blind assessment’

Asking for assistance

More than

just a grade

3.3 Toward a fair assessment culture

The sandwich feedback technique


4.1 The process of testing

Process


4.2 Teaching objectives

Interrelated levels


23

How are the following test formats related to general educational goals:

� Pattern drills: what kind of student personality is fostered if a teacher prefers pattern drills?

� Interpreting Shakespearean sonnets: what kind of educational goals are implied if a teacher enjoys the interpretation of ‘great works of art’?

� Writing your CV and preparing for a job interview: what kind of general educational goals are implied if a teacher focusses on career-oriented skills?


4.2 Teaching objectives – evaluation sheets


Oral performance

cf. Haß et al. 2008: 278

Writing performance

cf. Haß et al. 2008: 279

4.2 Teaching objectives – designing tests

25

Purpose

Type of learner(s)

Types of skills

Language functions

Text types

Task types

Instructions

Evaluation

& grading


cf. Weskamp 2001: 174

4.3 Typology of tests and tasks – task types


4.3 Typology of tests and tasks – advantages and disadvantages of task types


4.3 Typology of tests and tasks – task types for oral tests


Eisenmann & Summer 2012: 425-26

4.3 Typology of tests and tasks – examples

Closed task (Kieweg 1999b: 18)

Frank: Do you know Mrs Miller? – Peter: Of course, I do.

� We know each other for almost

two years.

� We’re knowing each other for

almost two years.

� We’ve been knowing each other

for almost two years.

� We’ve known each other for almost

two years.

Open tasks

� Write an essay on ‘Beauty.’

� Write a letter to the editor of a newspaper, responding to an article on the death penalty.

� Critically discuss the elements of the American Dream (e. g., with reference to the tragedy Death of a Salesman).

� Conduct an interview at an international company, asking international employees about their jobs. Present your findings in class.


5.1 Grading scales

305. Grading

5.1 Grading scales – German high school diploma (Abitur)

Discuss which criteria should be considered as having more importance, which less. How clear is the difference between the level of points? (TMBWK 2010: 2-4)

315. Grading

5.2 Alternative assessment formats

Focus on interaction and real-life tasks

Testing of interactional skills

Evaluation of cooperation in group work

Evaluation of media skills

Self- and peer-assessment

Assessment of the learning process

325. Grading


Errors and mistakes: deviations from language norms, part of the learning process

� Mistake = lapse in performance

� Error = gap in learner’s knowledge (competence error)


6. Correcting mistakes – teachers

34

Sensitive

Priorities

Pre-activities

Editing techniques

Peer-review

Keep track

of typical

mistakes


Reflect on and discuss the following statement:

A paper which is excessively marked and scribbled over by the teacher is no longer the student’s property. It becomes the teacher’s.

Peñaflorida 2002: 345


Correcting oral mistakes

Mistakes are the portals of discovery.

– James Joyce


Pre

sen

tati

on

an

d

pra

ctic

e

form &

accuracy Pro

du

ctio

n a

nd

co

mm

un

icat

ion

content & meaning


Correcting oral mistakes


Recasts

Clarification request

Elicitation

Postponed correction


Discontinue malpractices

376. Grading

Imprecise, unsystematic, unclear grading

No / only negative feedback

Taking too much time

‘Bleeding’ students’ papers to death

Negative view of errors and mistakes

based on Peñaflorida 2002: 344-45, 352; Jürgens & Sacher 2008: 70


Grade 7:

Define the three parts: closed, semi-open, or open tasks?

What is being tested and how is the overall grade computed?

How could design and grading be improved?



Grade 7:

Consider the grading and how it could be done differently.

Which grade would you have given, considering content, language, and form?

What kind of comments would have been more helpful?



Grade 10:

How do you consider the mistakes in the answer (graded and not graded)?

How would your grading differ if the CV had been practiced extensively before the test?


Recommended reading

Arendt, Manfred (2006). Beurteilung mündlicher Leistungen. Eine Untersuchung. In: Praxis Fremdsprachenunterricht; Part 1: 3.3, 3-10; Part 2: 3.4, 3-8.

Bebermeier, Hans (1999). Neue Formen der Leistungsfeststellung. In: Der Fremdsprachliche Unterricht Englisch 37, 46-51.

Eisenmann, Maria (2008). Formen mündlicher Leistungsmessung im Fach Englisch. In: Praxis Fremdsprachenunterricht 5.4, 26-30.

Huerta-Macías, Ana (2002). Alternative Assessment: Responses to Commonly AskedQuestions. In: Jack C. Richards & Willy A. Renandya, eds. Methodology in Language Teaching: An Anthology of Current Practice. Cambridge et al.: Cambridge University Press, 338-43.

Jürgens, Eiko (2000). Brauchen wir ein pädagogisches Leistungsverständnis? In: Silvia-Iris Beutel & Witlof Vollstädt, eds. Leistung ermitteln und bewerten. Hamburg: Bergmann und Helbig, 15-25.

Thaler, Engelbert (2008). Klassenarbeiten – eine Prozessperspektive. In: Praxis Fremdsprachenunterricht 5.4, 6-10.


Acknowledgments

Bohl, Thorsten (2009). Prüfen und Bewerten im offenen Unterricht. 4th ed. Weinheim et al.: Beltz.

Eisenmann, Maria & Theresa Summer (2012). Oral Exams: Preparing and Testing Students. In: Maria Eisenmann & Theresa Summer, eds. Basic Issues in EFL Teaching and Learning. Heidelberg: Winter, 415-28.

Finkbeiner, Claudia (2012). Introduction: Assessment and Testing and the Issue of Equity. In: Maria Eisenmann & Theresa Summer, eds. Basic Issues in EFL Teaching and Learning. Heidelberg: Winter, 385-401.

Haß, Frank; Werner Kieweg; Margitta Kutty; Andreas Müller-Hartmann & Harald Weisshaar, eds. (2008). Fachdidaktik Englisch: Tradition, Innovation, Praxis. Stuttgart: Klett.

Jürgens, Eiko (2000). Brauchen wir ein pädagogisches Leistungsverständnis? In: Silvia-Iris Beutel & Witlof Vollstädt, eds. Leistung ermitteln und bewerten. Hamburg: Bergmann und Helbig, 15-25.


Acknowledgments

Jürgens, Eiko & Werner Sacher (2008). Leistungserziehung und pädagogische Diagnostik in der Schule: Grundlagen und Anregungen für die Praxis. Stuttgart: Kohlhammer.

Kieweg, Werner (1999b). Klassenarbeiten: Überprüfung der grammatikalischen Kompetenz und Performanz. In: Der Fremdsprachliche Unterricht Englisch 33.37, 18-25.

Peñaflorida, Andrea H. (2002). Nontraditional Forms of Assessment and Response to Student Writing: A Step toward Learner Autonomy. In: Jack C. Richards & Willy A. Renandya, eds. Methodology in Language Teaching: An Anthology of Current Practice. Cambridge et al.: Cambridge University Press, 344-53.

Thüringer Ministerium für Bildung, Wissenschaft und Kultur (2010). Abiturprüfung 2010: Leistungsfach Englisch (Haupttermin) – Teil B. Erfurt: TMBWK, 1-9.

Weskamp, Ralf (2001). Fachdidaktik: Grundlagen & Konzepte. Berlin: Cornelsen.


Acknowledgments

The cartoons at the beginning of each ppt were designed by Frollein Motte, 2014. If not otherwise indicated, the copyright of the figures lies with the authors. The complete titles of the sources can be found in the references to the units unless given below. All of the websites were checked on 10 October 2015.

� Slide 11: http://bhsindo.erudisi.com/wp-content/uploads/sites/34/2015/08/child-865116__180.jpg

� Slide 12: http://blogs.rowlandhs.org/users/mattcole/weblog/2eadc/images/13571.jpg

� Slide 30: https://pixabay.com/de/b%C3%B6rse-b%C3%B6rsenparkett-business-pfeil-295648, http://blogs.rowlandhs.org/users/mattcole/weblog/2eadc/images/13571.jpg


12. assessment and evaluation · 2015-10-22 · presentation and practice ... chapter 12:...

Documents