ppg module tsl3105 topic 4 assessing l&s skills

TSL 3105 Teaching of Listening and Speaking Skills in the Primary ESL classroom

TOPIC 4 ASSESSING LISTENING AND SPEAKING SKILLS

4.1 SYNOPSIS

Topic 4 introduces you to the Assessment of Listening and Speaking Skills. It provides an overview of the important principles that a teacher has to bear in mind when assesing listening and speaking skills. You will also be introduced to the differences between teaching and testing, and develop the ability to distinguish the differences between accuracy and fluency based tests of listening and speaking. It also aims to help you develop a better and clearer understanding of the assessment strategies related to the testing of listening and speaking skills that we most often pay little heed to. The conventions of spoken language and the factors that affect listening and speaking skills are also included in this section. You will also be introduced to the importance of giving effective feedback and support to improve overall performance of the students.

4.2 LEARNING OUTCOMES

By the end of Topic 4, you will be able to:

Identify and deliberate on issues related to assessing listening and speaking

Differentiate between teaching and testing

Distinguish the difference between accuracy and fluency based tests of

listening and speaking

Develop a clearer and better understanding of assessment strategies related

to the testing of listening and speaking skill most notably the communicative

testing listening and speaking

Develop a better understanding on giving effective and construtive feedback

and support to improve performance

lgp/wsl/kj 2012


4.3 ISSUES IN ASSESSING LISTENING AND SPEAKING

Comment on the following statement:

“The best teacher is the teacher who devises classroom methods and techniques that derive from a comprehensive knowledge of the total process of language learning, of what is happening within the learner and within the teacher and in the interaction between the two. All of this knowledge, however, remains somewhat abstract in the mind of the teacher unless it can be empirically tested in the real world.”

H.D. Brown (1987:218)

a. Answer True or False:

• Tests involve students and teachers only.

• Tests are carried out on a weekly, monthly and yearly basis.

• A test is a form of measurement.

• All tests are formal.

• Questions asked by the teacher about what has been learnt in a lesson can be considered a ‘test’.

Revision:

Four basic criteria to devise, use or adapt tests of listening and speaking

• Validity

• Reliability

• Practicality

• Washback Effect/ Instructional Impact

Validity

• We must make sure that we are testing what we are teaching and what the students want to be learning. Only then is the test fair and appropriate

lgp/wsl/kj 2012


• A test that measures what it is intended to measure is called a “valid” test

• We need to determine in advance what we want to measure (test)

• We must then design items, tasks, or prompts for the test that measure what we want to measure

• For example: A student who has not been taught prepositions is tested on prepositions --> the test can be viewed as lacking in validity because it tests what the student has not learnt

Reliability

• We have to be sure that a test or an assessment procedure is reliable.

• Reliability is concerned with consistency

• For example: if you tape record your students speaking in English and ask another teacher to evaluate the students’ speech using a ten-point scale, your evaluation and the other teacher’s evaluation should not differ greatly

• Rater reliability Intra (the rater himself) and Inter-rater (between raters)

Can you think of some factors that affect intra-rater and inter-rater reliability?

What can you do to minimise the variability (differences) that exist?

Practicality

• Practicality refers to the fact that a test or other assessment procedure can only be useful if it does not make unreasonable demands on resources, including time, money and personnel

• For example: Interviewing each student for thirty minutes (very thorough way to assess but what if you have 100 students?)

Washback Effect/ Instructional Impact

• The effect a test has on teaching and learning

• Does the test encourage people to prepare for speaking tasks, or does it cause them to study grammar rules or obscure vocabulary items?

• Washback can be positive or negative

• Positive promotes the development of the skills or knowledge to be learned

lgp/wsl/kj 2012


• Negative hinders the development of the skills or knowledge to be learnt

• For example: Listening and speaking skills are neglected in schools mainly because the two skills are not tested at the national level and more importantly, the grades or marks for the testing of listening and speaking skills are not included in the overall results. Teachers therefore view the teaching of listening and speaking as an unnecessary thing in schools. Hence, the over emphasis on the reading and writing skills and a neglect of the listening and speaking skills in schools (Negative washback effect).

Can you think of some other instances of positive and negative washback effects of tests on the teaching and learning process in schools?

b. What is a test?

• A yardstick a teacher uses to measure the performance of a student (Nesamalar et. al., 2005)

• A method of measuring a person’s ability or knowledge in a given area (H.D. Brown, 1987)

• We test every day in virtually every cognitive effort we make – when we read a book, listen to the news, or prepare a meal, we are testing hypotheses and making judgments

We will now deconstruct the definition of a test by H.D. Brown, 1987 to develop a better understanding of what a test really is:

“A method of measuring a person’s ability or knowledge in a given area”

A Test is a “method”

• There is a set of techniques, procedures, test items

• The method generally requires some performance or activity on the part of either the testee or the tester, or both

• The method may be intuitive and informal or explicit and structured

A Test is a method of “measuring”

• A test has the purpose of ‘measuring’

lgp/wsl/kj 2012


• Formal and informal tests nature of the quantification of data

• Informal tests intuitive and difficult to quantify; judgment rendered in global terms (good, poor, fair etc.)

• Formal tests quantification is important: using carefully planned techniques of assessment; for comparison within individual or across individuals

A Test…measuring “a person’s” ability or knowledge

• Need to understand who the testees are?

– Their previous experience

– Their entry behaviour

• Is the test appropriate for the testee?

• How are scores to be interpreted for individuals?

A Test …measuring a person’s “ability or knowledge”

• Competence

• A test samples performance but infers certain competence

• A driving test for a driver’s licence a sample of performance to infer general competence to drive a car

• A language test samples language behaviour to infer general ability in a language

A Test…ability or knowledge “in a given area”

• Proficiency test actual performance involves only a sampling of skills, that “area” is overall proficiency in a language

• A pronunciation test might test only a particular sound/ a phonemic minimal pair

4.4 TEACHING VERSUS TESTING

• Tests serve many of the needs of teaching

lgp/wsl/kj 2012


• To monitor progress as we teach

• Topic tests used to assess how well a student has mastered what has been taught

• To find out specifically what areas pose difficulties for students diagnostic tests

• Testing and teaching are very closely interrelated but do not have the same focus

• Tests assess the products of learning (student’s previous learning)

• Teaching materials, methodology and the classroom enable students to succeed in the process of learning (prepares students for current and future learning)

• Primary function of teacher to ensure learning takes place; tests are one of the tools to help him do this job well;

• A test is only a sample of what the student is supposed to know – a test is supposed to pick out the most important aspects of the skill(s) that have been taught

• A test is also often used as a guide as to what would be the most important things to teach a test influences what is taught (backwash effect)

• A test often leaves out certain important skills because of practical constraints

• UPSR & PMR no Listening and Speaking component; therefore teachers do not pay sufficient attention to these important skills

• What is taught should be decided by reference to how important the skills are to a child’s present or future life

• Teachers should not just teach students only those things that will help them pass exams

• Some teachers use the exam formats for teaching purposes this is WRONG!!

• A test usually seeks the most economical way of finding out what the student already knows; leaves out many things that are important

• Using MCQs for speaking tests (not suitable)

• Teaching should keep in mind the real reasons for learning and the real circumstances in which the language or skill would be used

lgp/wsl/kj 2012


Discrete Point vs Integrative Testing

• Discrete point tests based on the assumption that language can be broken down into component parts and tested adequately (listening, speaking, reading, writing, phonology/ graphology, morphology, lexicon, syntax)

• Discrete point approach met with criticism in the “integrative sociolinguistic” era where the emphasis is on communication, authenticity, and context.

• John Oller (1976, 1979) argued that language competence is a unified set of interacting abilities which cannot be separated apart and tested adequately

• Communicative competence is so global and requires integration that it cannot be captured in additive tests of grammar and reading and vocabulary and other discrete points of language

• “If discrete items take language skill apart, integrative tests put it back together. Whereas discrete items attempt to test knowledge of language one bit at a time, integrative tests attempt to assess a learner’s capacity to use many bits all at the same time” (Oller, 1979:37)

Example of a Discrete Point Test

• A typical proficiency test with MCQs divided into grammar, vocabulary, reading etc.

Examples of Integrative Tests

• Cloze tests – Oller (1976, 1979) good measure of overall proficiency

– requires a number of abilities that lie at the very heart of language competence

– Knowledge of vocabulary, grammatical structure, discourse structure, reading skills and strategies and an internalized “expectancy” grammar

• Dictation – a potentially appropriate integrative test

– Taps into certain grammatical and discourse competencies

– Requires careful listening, reproduction in writing of what is heard, efficient short-term memory, and some expectancy rules

lgp/wsl/kj 2012


Discrete Feature Listening Tests

• Auditory Discrimination

– distinguishing specific sounds from a background of different sounds

• Minimal pairs – sound discrimination test

– Pen or Pan; Pill or Peel; Van or Ban

• Identifying words that rhyme

• Identifying intonation patterns

• Identifying stressed and unstressed syllables

Discrete Feature Speaking Tests

• Minimal pair tests

– /pen/ & /pin/; /bell/ & /bill/

• Intonation tests

– He is here.

– He is here!

– He is here?

– Here he is!

• Reading aloud

– A test of pronunciation, stress and intonation

• Tests of language functions and their linguistic realizations

– Writing down what is missing in a conversation

Task-based Listening Tests

• Listening tasks that students normally engage in – authentic tasks

• Listen to a text/ description and answer questions/ identify places on a route map

lgp/wsl/kj 2012


Task-based Speaking Tests

• Conversational exchanges – different degrees of control (controlled, partly controlled, one-sided dialogue, incomplete dialogue with prompts)

• Using pictures – single picture/ object, pictures for comparison, a series of pictures to make up a story

• Oral interviews – common oral interaction test in which one or two testers interview a candidate regarding a set of pre-determined topics

Assessing Listening and Speaking Skills

• Do we use discrete point tests or integrative tests to assess L&S skills?

• Or do we use a Communicative Test?

Communicative Test

• To test the communicative production of language

• A communicative test has to meet some stringent criteria. It has to test for:

– Grammatical competence

– Discourse competence

– Sociolinguistic competence

– Strategic competence

– Illocutionary competence

(Wesche 1983, Swain 1984)

• Has to be pragmatic

• Learners have to use language naturally for genuine communication

• Use authentic language within a context

• Should be direct

• Should test learners in a variety of language functions

• DIFFICULT TO MEET ALL THESE CRITERIA!!!

lgp/wsl/kj 2012


Communicative Tests - Merrill Swain (1984)

Four primary criteria

• 1. Start from somewhere tests should build on existing knowledge & principles

• 2. Concentrate on Content have interesting motivating and substantive content; integrated and interactive

• 3. Bias for best need to do everything possible to elicit the very best from students

• 4. Work for washback we should not teach toward a test but we can use tests as teaching tools; feedback devices

4.5 ACCURACY VERSUS FLUENCY

• Activities which focus on accuracy try to get students to say something correctly (correct grammar, correct word form).

• Activities which focus on fluency try to get students to communicate successfully, even if they make some mistakes.

• When using communicative tests, the teacher constantly faces the dilemma of deciding which is the focus of the assessment: accuracy or fluency or both

• Both are equally important for language learning and language use.

lgp/wsl/kj 2012


• Accuracy certainly helps students communicate effectively and efficiently, and more importantly, they may need a high level of accuracy to pass exams.

• Fluency activities are important because they allow students to express their ideas and communicate in a meaningful and enjoyable context.

REFLECTION

As a teacher, which would you consider more important when teaching: accuracy or fluency?

As a teacher, which would you consider more important when testing: accuracy or fluency?

Can you think of an activity that focuses on accuracy which can be used to test L&S skills

o What will the focus of the activity be?

o Grammatical accuracy?

Can you think of an activity that focuses on fluency which can be used to test L&S skills?

o What will the focus of the activity be?

o Ability to communicate? What about errors made?

TUTORIAL TASK

What is validity and how does it differ from reliability?

Why is practicality an important issue in learners’ listening and speaking skills?

What are positive and negative washback? Provide a few examples of positive and negative washback effects of testing on teaching.

Think about an important listening and speaking test that you have taken. How did you prepare for that test? What did your teacher do to help you prepare for the test? Did that test have a positive or negative impact on you? Describe the test that you took.

If you have not taken such a test before, think of possible reasons why it was not carried out.

lgp/wsl/kj 2012


4.6 ASSESSMENT STRATEGIES FOR LISTENING AND SPEAKING SKILLS

Types of activities that can be used for testing L&S communicatively

• Reaching a consensus (Groups of 3/4)

– Deciding on ten out of twenty items that they would take on a 2-week trip to New York; need to provide justifications

• Moral Dilemmas (Groups of 3/4)

– Invigilating an important exam

– See a student cheating with notes

– 4 possible action: ignore, warn, ask student to leave, report to authorities

• Discussion

– Using controversial topics (smokers should be banned, forced to quit habit, jailed, allowed to smoke in designated areas)

– Debates (two sides argument: points for or against); Balloon debate who to throw out of a balloon which is losing air!

• Relaying instructions

– Giving each other instructions

– Making models

– Describe and draw

• Communication Games

– Find the differences between two pictures

– Describe and arrange

– Story reconstruction

• Problem Solving

– Desert Dilemma

• Simulation (reality of function, a simulated environment, structure) and role play (pretending to be someone they are not)

– Creating the pretence of real-life situations in the classroom; ‘simulating’ the real world

lgp/wsl/kj 2012


– To provide practice in real world use of English

Devising Marking Bands - Descriptors

• When assessing L&S skills communicatively, what are the sub-skills or areas you will be looking at?

– Topics to be covered?

– Activity type? Individual? Pair? Group?

– Duration?

– Holistic or analytical marking?

– Accuracy? Grammatical?

– Fluency?

– Pronunciation?

– Appropriacy?

– Message conveyed?

– Others?

• Depending on what your assessment objectives are, you may choose to incorporate a number of sub-skills or areas listed above in your marking band.

• The marking band is formulated to address issues related to reliability – inter-rater variability (the inconsistencies between markers) so that a more objective score can be obtained by all raters.

4.7 THE SKILLS OF PROVIDING CONSTRUCTIVE FEEDBACK

• What is feedback?

• Why feedback?

• How to provide constructive feedback?

• Exercise?

• Innovations in methods of feedback?

• Take home message?

lgp/wsl/kj 2012


What is a feedback?

Why Feedback?

• Provision of important information to the pupil.

• A stimulus for further learning and training.

• Show pupil the level of their performance.

• Address the weaknesses and deficiencies of the pupil.

• Decide the progress of the pupil.

• An effective method to evaluate the whole program.

lgp/wsl/kj 2012


Characteristics of Good Feedback

Constructive Feedback

• Constructive feedback can help the trainee to improve in their knowledge, skills and attitudes

• It can help them to rate their learning in a realistic way

• It can help them to be more self-regulated about their learning

• It should be focused on behaviour rather than the person, and on observations rather than inferences or judgments

Appropriate Feedback

• Feedback should be completed as soon as possible after the event, before

they forget details of the events

• Confidentiality and privacy should always be respected

• Check if the feedback has been understood

• More than one approach is preferred (e.g. visual and verbal in the same time)

4.9 HOW TO PROVIDE AN APPROPRIATE CONSTRUCTIVE FEEDBACK

lgp/wsl/kj 2012

Good Feedback

CONSTRUCTIVE APPROPRIATE


A FEEDBACK SANDWICH

A feedback which starts with a positive statement,

followed by a negative statement and finished

with a positive statement

POSITIVE -----> NEGATIVE -----> POSITIVE

Recommended feedback techniques in giving feedback

• Create a respectful, friendly, open-minded unthreatening climate

• Elicit thoughts and feelings before giving feedback

• Be non-judgmental

• Focus on behaviour and specific observed facts

• Give right amount of feedback

• Suggest ideas for improvement

• Base feedback on well-defined, negotiated goals

• A constructive & appropriate feedback is an essential tool to improve performance

Some Questions To Think About

• Look at an existing language test (listening and speaking), evaluate it in terms of the various criteria for a communicative test:

– Is the test communicative? Why do you say so?

lgp/wsl/kj 2012


– Does it focus on both the listening and speaking skills?– What is the focus of the test --> accuracy or fluency or both?– Is the activity suggested appropriate?– Does it truly measure the pupils’ ability to communicate?

• Is there any way to resolve the dilemma of giving large-scale communicative tests and still maintaining a sense of practicality (the feasibility of scoring thousands of tests relatively quickly and cheaply)?

• Computers are revolutionizing the testing industry. Can computerized tests meet communicative test criteria? In what way can the addition of an interactive video component to a computerized test help to meet those communicative criteria?

References:

Brown, G., & Yule, G. (1983). Teaching the Spoken Language. Cambridge: Cambridge University Press.

Brown, H.D. (1994). Teaching by Principles: An Interactive Approach to Language Pedagogy. Englewood Cliffs, NJ: Prentice Hall Regents.

Brown, H.D. (2004). Language assessment Principles and Classroom Practices. Longman.

Celce-Murcia (Ed.), Teaching English as a Second or Foreign Language (2nd ed.) (pp. 81-106). Boston: Heinle and Heinle.

Hughes, A. (2003). Testing for Language Teachers. Cambridge: Cambridge University Press.

Nunan, D., & Miller, L. (Eds.). (1995). New Ways in Teaching Listening. Alexandria, VA:

Penny Ur. (1996) Teaching Listening Comprehension. Cambridge: Cambridge University Press.

Weir, C.J. (1990). Communicative Language Testing. Prentice Hall.

lgp/wsl/kj 2012

ppg module tsl3105 topic 4 assessing l&s skills

Documents