methodology testing

8/14/2019 Methodology Testing

1/18

Anca Cehan 1

Catedra de Limba si Li teratura engleza

14. TESTING AND EVALUATION

Testingis often a misunderstood word. When hearing it, many people envisageformal written tests, done by pupils working on their own to a time limit. In fact, we test

our pupils ability to speak write or read, or their listening comprehension skills everyday throughout the lessons. However, because marking written tests is easier thanmarking oral tests, and because written tests take less time and are easier toadminister, most tests are written.

Whenever we checkthat a pupil is understanding, following, making progress,assimilating a new word, pronouncing a correct sound, we are testing our pupils. Also,whenever we informally assess whether our teaching is effective, we are testing ourpupils.

By the end of this unit, you will be able to:

explain the difference between various types of tests justify the use of tests in your classes

discuss what communicative testing is

devise specific tests for your classes.

1 Informal and Formal Testing

Broadly speaking, there are two kinds of classroom testing: informal andformal.Informal testing is usually done orally and has a short-term objective within a lesson.Formal testing is usually done via the written medium, and tends to have more long-

term objectives. Formal testing divides into three categories.

1.placement testing, i.e. finding out what level the pupil has reached incomparison with an objective scale of competence, in order to put him/her in a suitableclass;

2.diagnostic testing,i.e. finding out what the pupil needs;3.progress/achievement testing,i.e. finding out what the pupil has learnt.

Placement and diagnostic testing are often carried out in unison when a pupilfirst enters a school. Progress tests reflect the work of a lesson (e.g. homework) or thework covered in a week or a longer period of time.

1.1 Informal TestingInformal testing refers to the techniques we use in a lesson to keep a constant

check on our pupils minute-by-minute progress and the effectiveness of our teaching. Itcan be done using concept questionsto check assimilation of meaning, by eliciting*andby peer correction*.

Here are a few types of informal tests:

asking concept questions

checking understanding of instructions

eliciting pupils own examples

eliciting pupils explanation of a grammar rule eliciting pupils definition of a vocabulary item

controlled oral practice (e.g. drills)


2/18

Anca Cehan 2

elicited/cued*/guided dialogue-building

elicited/cued/guided monologue-building

free stage activities

language games

communication games

comprehension questions

tasks based on listening/reading material

As a result of informal testing, both your pupils and you get an ongoing andimpressionistic idea of their progress.

1.2 Formal TestingIn formal testing, you give a certain time to a group of pupils to do a test that

was previously prepared, which is then corrected. Formal tests are not necessarily setby you, the class teacher, and they may not be related specifically to previousclassroom teaching. Apart from their grade or mark, the pupils may never see the testsagain.

Formal tests can be most types of written exercises:

transformatione.g.: He's a fast runner. He runs...

clause combinatione.g.: She had a cold. She went swimming. (although)

gap-fillinge.g.: The chocolate cake looks so good. It must be very...

multiple-choicee.g.: Which of the three solutions is closest to the given word: hostagea) prey b) victim d) captive

sentence-completion, etc.e.g.: My friend would be all right if....

SAQ 1

What other types of written activities could be used informal written testing?

The written work that the pupils produce provides a basis for assessment andanalysis.

2. Approaches to Testing

Traditional types of formal tests may test linguistic competence to some extent,but they do not test linguistic awareness or communicative competence. Thus, we canidentify three different approaches to testing:


3/18

Anca Cehan 3

discrete item testsThese test individual grammar rules or vocabulary items (e.g. Jane (go) to

school by bus every day.)

integrative testsThese test a more global linguistic awareness, e.g. cloze tests where every nth

word is deleted from a passage.

communicative testsThesetest the pupils communicative effectiveness in each of the four language

skills, and also the socio-cultural awareness of language choice in a range of contexts.

Most teachers believe that their job is a constant process of TESTING (orASSESSING) TEACHING TESTING AGAIN... and so on. In fact, pupils work off thefeedback they get from their teachers and teachers work from the feedback they getfrom their pupils. Thus teachers can continuously assess individual pupils progress andthe effectiveness of their teaching.

There are four basic issues in testing:

the purpose of the test

the criteria for deciding whether the test is good or not

what should be tested

what testing techniques should be used.

2.1 What Does a Test Measure?A test is a measuring device, and a means of comparison. We use a test when

either we want to compare a pupil with other pupils who belong to the same group, orwe may be comparing what the pupil can do now with what s/he has done in a past test.We may therefore use a test to

a. compare a pupils present performance with his/her past performance. We dothis using a progress /achievementtest, based on what has been taught.

b. find out if a pupils language abilities compare with the minimum requirementsfor a task (e.g. studying in an intensive English class, which requires a certain level ofEnglish). In such circumstances, we use a proficiency test, based on what the pupilneeds to be able to do with the language to perform a certain task.

c. find out which of the classes available is best suited to the pupils needs. Thisis a placementtest and is most often used on entry into a school.

d. find out what a group of pupils specific needs are, what they are good at orweak on; to see which skill areas need more work, and what kind of work. This is doneusing a diagnostic test, based both on what the pupils should know and on what maystill need to be covered. A diagnostic test is similar to a placement test in content but itmay be a more precise instrument. Actually, we may consider that informal diagnostictesting is going on for some time at the beginning of a course.

e. find out which are the best pupils in a group. This is competitive assessmentvia a selectiontest.

f. find out the language learning abilities of a pupil or group of pupils, to see towhat extent they would benefit from a language course. This is called an aptitudetest.

With the exception of aptitude tests, all the tests overlap to a certain extent interms of content language and its use. However, each one has a different aim and adifferent relationship to teaching. These differences influence the content of a test, the


4/18

Anca Cehan 4

way in which it is handled and marked, and who writes the test. The three tests thatmost concern us are the progress, diagnosticand placementtests.

2.2 Progress TestsThese are the tests that you administer most often, and which affect your pupils

learning directly and immediately. Their aim is to find out if what you have taught has

been assimilated sufficiently to be used accurately and appropriately, or whetherremedial work is needed. Progress tests are based on what you personally have taughtyour pupils, on what the syllabus or the textbook directed you to teach. That is why,progress tests are, in a sense, retrospective to teaching. They act as a kind of summaryof a number of hours work in class or at home.

Regular progress tests can help you and your pupils to see improvement incertain areas or skills over a longer period. This is particularly useful with moreadvanced pupils where progress is often more difficult to see. Progress tests are also agood way of indicating to pupils that more effort is required of them if they are to reach acertain standard.

We should consider the results of progress tests in conjunction with continuous

assessment. As some test results may indicate a bad day for some pupils, you may alsoneed to consider factors such as: how hard they have worked, how much passiveknowledge they have, whether they like a challenge or not, and perhaps even the groupspirit.

A progress test, administered at the end of a year or at the end of a term, hasthe aim of helping you and your pupils to see progress or lack of it. Such tests areprobably best written by someone with an overall view of the different levels and of howthey relate to one another and the syllabus. Many textbooks offer such tests at the endof the year. However, if the aim of the progress test is to summarise a weeks work or todraw together the different elements of a unit, you are the best person to write the test.Such a test needs to be strictly related to what you have taught that class of pupils. Theresults of such a test have the advantage of not only telling your pupils and yourself howmuch they have learned, but also what needs remedial work in subsequent lessons. Ifthe test exposes huge gaps in your pupils knowledge, it has the function of a diagnostictest as well.

However tests can go wrong, not only because of the pupils lack of knowledge,but also because of problems in the writing of the tests themselves. When you writetests, bear in mind these five rules suggested by Harmer (1987, pp. 58 - 59):

1. Dont test what you havent taught . Unless you are testing reading orlistening comprehension, you only test the language you have exposed your pupils to.

However, if you ask your pupils to write freely, then encourage them to show as muchas they know.

2. Dont test general knowledge. Remember that you test onlytheir English.

3. Dont introduce new techniques in tests. For instance, ask them to do asentence-ordering activity, only if they have worked with jumbled sentences before.

4. Dont just test accuracy. A progress test should examine the pupils abilityto use language, not just their grammatical accuracy. Give them the opportunity toexpress themselves freely, too.

5. Dont forget to test the test. Show it first to a colleague, who might identify

problems you have not thought of, such as unclear instructions, mistakes, or thedifficulty of the test (too high or too low). If possible, try your test out with a similar classor a class of a slightly higher level.


5/18

Anca Cehan 5

2.3 Diagnostic TestsWhile progress tests are based on what you have taught, diagnostic tests are

based on what you think needs to be taught. Diagnostic tests are usually set at thebeginning of a course when you want to know your pupils.

Based on the information provided by a diagnostic test, you will plan yourlanguage work for the class or group. A diagnostic test on the first day will help you to

decide which areas need remedial work, and which ones the pupils have at least someknowledge of. This information will enable you to select the most necessary languagework and the right approach to input, practice and skills work. This is particularlyimportant at intermediate level and above, when learning is a process of constantremedial work in order to help pupils achieve increasing mastery of complex aspects oflanguage use.

You may even use the same diagnostic test at the end of the course, as aprogress test, so that both you and your pupils see how and where they have improved.

2.4 Placement TestsPlacement tests are concerned with the general needs of a pupil, and are based

on a general assessment of his/her ability to use English. When testing a pupil forplacement purposes you need to bear in mind an image of the syllabus at all levels, anda knowledge of what demands are made on pupils at different levels. Based on what thepupil understands and produces in the light of this criterion, you place him/her up ordown the scale.

Placement tests are important for you, as you have to cope with the results ofplacement tests, even though you may not be involved in designing them.

3 Assessing Tests

Four main issues concern us in assessing a test:

validity

reliability

scorability

administrability

A test has validity when it does what it sets out to do, and does not test otherthings unnecessarily (such as memory, intelligence, non-linguistic knowledge,personality, etc). A valid test uses means of testing appropriate to the aims of the test.

A reliable test will give consistent results. If we administer the test again to the

same pupils or to other pupils at the same level, the test will give the same results. Togive reliable results, the test should be long enough.

A scorable test is quick to score, and not very time-consuming.An administrable test is easy to administer.

3.1 Reliabil ity and Valid ityIt is important for the results of a test to remain the same if the test is marked by

different people, or by the same person at a different time, or if the same pupils did thetest again under the same conditions. Reliable results are also important if you want tocompare one group of pupils with another. Reliability is easier with written tests thanspoken ones.

What can you do in order to make sure that your tests are reliable?


6/18

Anca Cehan 6

a. Make sure the test is long enough, and if possible test the same things inmore than one way.

b. Pilot your test: try it out on a group and note any problems that emerge whenadministering the test.

c. Mark the test twice or give it to somebody else to mark it, to see if both of you

agree with the marking. The more people agree with your marking, the more reliableyour test will be.

Validity emphasises the importance of the pupil's reaction to the test. It is veryimportant that the test looks like a good one to the pupils; otherwise, they may not feellike putting in the necessary effort to do it well. This means that we should design teststhat are clearly connected with what we have done in our class, in terms of both testcontent and testing technique. Our pupils' previous experience of tests will affect whatthey consider to be a valid test or not. Our main aim is to assess their languageknowledge and skills, and this will be difficult to do if the pupils are unsure how tohandle a testing technique, multiple choice, for example. it is, therefore, in everyone's

interest to design tests that do not cause pupils undue stress.The secret of content validity is to:

a. make sure you are very clear in your mind both what you want to test andwhat different testing techniques you demand of the pupils;

b. analyse the results of any test you write to see which test items got theanswers you intended them to get. Then you can consider what happened and modifythe test items for the next time.

A reliable test is not necessarily a valid one and vice versa. For example, if youwant to find out if your pupils can write an essay, then the valid test is to ask them towrite an essay. However, if you have not thought carefully about the degree of guidanceyour pupils need to receive in relation to what you expect, and about how you will markthe test, this can be an unreliable way of testing essay-writing skills.

On the other hand, a test with multiple-choice questions may be a reliable wayto test pupils ability to recognise the correct structural item, but it is not a valid way oftesting the pupils ability to produce these items.

SAQ 2

Below are five examples of test items that have the aimof testing pupils' ability to produce a simple past question using

How.Which would you say are valid ways of doing this?

a) Make a question from this sentence:He went to Brighton by car.

b) Make the question for the answer below:

............................ to Brighton last year?

.I went by car.

c) Make a question about the phrase underlined:I went to Brighton by car.

d) Make the question for the answer below.By car.

e) Make this sentence complete: How/go/Brighton?


7/18

Anca Cehan 7

3.2 Scorabilit y and Administ rabilityAdministrability refers to how easily a test can be made or done. A number of

technological developments, such as the photocopiers, the audio-recording equipmentand the computers have made testing an easier task for the teachers.

There is a tendency for the more reliable tests to be easier to mark. Where it ispossible to be more objective about the answer, it is easier to mark. For instance, it is

easier to mark multiple-choice questions, because the pupils' answers can be tightlycontrolled and anticipated, and there is room for one right answer only. By contrast, in aletter, for instance, you need to think of the way the pupils have expressed themselves;you may need to reconstruct errors; you may be uncertain about the importance of anerror, etc. In other words, however much you think about the criteria for marking andattempt to standardise your marking, the unpredictability of free writing makes markingmuch more subjective. Therefore, we have two kinds of testing techniques in terms ofmarking: subjective and objective.

3.2.1 Marking TestsTests that are more reliable are easier to mark and thus it is possible for you to

be more objective about the mark.

SAQ 3

Which of the following is easier to mark and why? Apiece of free writing such as a thank you letter or multiple-choiceanswers? Write your answer (about 100 words) in the spaceprovided below.

There are two kinds of testing techniques to choose from in terms of marking:subjectiveand objective. These terms do not refer to the writing of the tests, but to thetype of marking they require. Actually, all tests are fundamentally subjective inconstruction because they reflect certain attitudes to learning and to what is learnt. Bothsubjective and objective techniques have their strengths and weaknesses.

Objective testsThese tests are very quick to mark. They are based on predicted answers and

on total control of what the pupil shows s/he can do. The big disadvantage of such testsis that we cannot find out if the pupils can do anything else.

Subjectivetests

These tests are easy to design and administer, but their marking can be a time-consuming process, involving a lot of decision-making about the quality andacceptability of the answers. In such tests, the pupils can avoid or get round things theyare not sure of or do not know. They can show what they can do beyond what the test ismeant to test. This could be as much to their disadvantage as to their advantage.

When setting out to write tests, remember to create a balance both for yourselfand for your pupils when marking. Try to devise ways in which subjective testing (e.g.free writing) can be put within a framework to restrict what the pupils produce (e.g.include the following points...or compare and contrast with...) to help you make yourmarking both easier and more standardised and reliable.


8/18

Anca Cehan 8

4 Discrete Item vs. Integrative Tests

SAQ 4

Consider the following two test items:

Test 1. Complete this sentence with a suitable word orphrase.

A: John failed his exams, you know.B: Well, it is his own fault, he harder.

Test 2. Read this passage and follow the instructions:

Your friend, John, had his first date last night. He playedfootball in the afternoon and didnt have time to have a bath. Hejust brushed his teeth, changed his clothes and rushed to thecinema. He was half an hour late. He had chosen a horror film,and after a few minutes, the girl asked to leave. John had anargument with her in the cinema, and then they left. They wentto a restaurant, and at the end of the meal, John told the girl sheought to pay half the bill. In the taxi on the way home, he tried tokiss her but she started crying.

Although it is too late, give John some advice about lastnight.

Now in about 60 words, answer these questions:a) What do these two test items have in common?

b) Which one tests one specific language item?

.

c) Which one demands more of the pupil in terms oflanguage skills?

..............................................................................................................................

d) Which one is closer to what happens with languagein real life?

.

Test item 1 is an example of discrete item testing, while test item 2 is anexample of integrativetesting.


9/18

Anca Cehan 9

Test item 1 isolates a particular item of language and keeps the surroundinglanguage to a minimum. Test item 2, on the other hand, is a small-scale example ofintegrative testing because the pupil has to demonstrate a wider knowledge of Englishand utilise an ability to read for gist.

The difference between discrete item and integrative testing is primarily one ofintention.When we want to know if a pupil can recognise or produce a specific item of

vocabulary or structure, a specific functional exponent or a certain pronunciationfeature, then we use discrete-item techniques. If we want to know how well a pupil cancombine her/his knowledge of grammar and vocabulary in skills work with a specificaim, then we use integrative testing techniques.

SAQ 5

Here are a few testing techniques you are familiar with.Say whether they are examples of discrete testing or ofintegrative testing, by writing D(discrete) or I(integrative) in thespaces provided:

essay

dictation

multiple choice

true/false

written answers to comprehension techniques

gap-filling

The discrete-item techniques have the advantage of making us considercarefully what we are testing.

One of the chief features of the communicative approach is to appreciate theimportance of language in use, and thus to give more emphasis to language skills as

the end, and language items as the means. Thus, communicative lessons provide abalance between items and skills. Traditional testing techniques have also been re-evaluated, and tests such as dictation have been reconsidered because in doing it,pupils have to show that they can integrate knowledge of the parts of the language inorder to demonstrate their language level and ability.

While it is recognised that some kind of integrative testing is more appropriate indetermining overall competence, this is not to say that discrete item testing techniqueshave been thrown out. However, they should be used as ways of testing specificlinguistic sub-skills or language items and not as means of assessing overallcompetence in a skill.

When we need precise information about the gaps in a pupils receptive or

productive knowledge of the linguistic systems, or about her/his difficulties with specificsub-skills which affect the overall competence in one of the four skills, then discrete itemtesting will come into its own. Discrete item testing is of most use for diagnostic andachievement / progress testing, where we are concerned with finding out what the gapsare or with how much of our teaching has been absorbed, or what the pupil can andcannot do.

SAQ 6

Would you also include integrative testing techniques ina diagnostic or a progress test? Why (not)?


10/18

Anca Cehan 10

Here are some common discrete item (and objective) testing techniques:

transformatione.g. Complete these sentences:

Hes a fast runner. He runs

e.g. Change the word in capitals to fit the sentences given:There were a lot of for the job. APPLY, etc.

e.g. Complete the second sentence so that it has the same meaning as the first:Im thirsty, she said. / She said

e.g. Make these sentences into questions:1. Ive got two sisters and a brother, etc.

insertione.g. Put the word in capitals into the right place in the sentence:She lives in an old farmhouse. HUGE

fill ine.g. Fill in the blanks with so, suchor such aand words from the text.

Jo was so annoyedwhen her sister got a Walkman that she didnt speakto her parents for days.

combinatione.g. Join these two sentences using although:

John had a cold. He went swimming.

sentence completione.g. My room would be all right if

re-arrangemente.g. jumbled words, sentence, paragraphsFind words/phrases in column B, which have a similar (in other tests, opposite)meaning to those in column A:

A Bdisappear moistfade leave

loathe lose colourdamp dislike intensely etc.

functional sentence matchinge.g. Match the sentence with when you use it:Could I come in there? persuadingIs there any way I could generalisingThey tend to be dark-haired interrupting, etc.

split-sentence matchinge.g. Combine these phrases to get sentences:

Would you mind my open the window?Would you mind if opening the window?Could I I opened the window?


11/18

Anca Cehan 11

skeleton sentencee.g. Make sentences:

This picture/paint/Van Gogh/long time ago

error analysis

e.g. Which part of the sentence is wrong?Im frightened/that youll/feel angry/to meA B C De.g. Correct the following:Do you like a cigarette?

situationse.g. You want a day off. How would you ask...a) your bossb) a colleague who works with you in a busy office.

4.1 Two Popular Techniques: Multip le Choice Tests and Cloze TestsBelow are described in more detail two commonly used testing techniques:

multiple choice tests and cloze tests.

4.1.1 Multiple Choice TestsMultiple-choice tests are examples of discrete item testing. Almost all written

exams rely on the multiple-choice technique because of its obvious scorability. It is usedprimarily for comprehension testing and vocabulary testing.

A test item consists of a stem and options. The options consist of the correctanswer and a number of distractors, e.g.:

STEM: I saw him yesterdayOPTIONS: see / saw / have seen / had seen

1 2 3 4

1, 3 and 4 are DISTRACTORS. 2 is the correct answer.The more distractors there are, the smaller the chances of pupils getting the

answer right by guesswork or luck. However, it is difficult to have a large number ofoptions that are not absurd.

The STEMmay be:

an incomplete statement, e.g.: I . him yesterday.

a complete statement, e.g.:He got over the flu in two weeks. a question, as in comprehension multiple choice.

Here are a few words of advice regarding options:

make sure that you test only one thing at a time

make sure all options are at the appropriate linguistic level of the pupils youwant to test

give only one correct option.

However, it is not impossible to have two correct options; for instance, two

correct options are possible in comprehension questions, rather than in questions thattest language elements. Even if this could reduce guesswork, make sure that allquestions have either two correct options or all have one.


12/18

Anca Cehan 12

do not include absurd options

avoid ungrammatical options

avoid ambiguity

to ensure face validity, have a few colleagues look it over your test.

Remember that it is useful to test major items twice in order to increase the

reliability of the score.

Here are a few advantages of multiple choice testing:

it enables objective scoring

it can easily expose language weaknesses

it encourages pupils to look at sentence context when making their choices

it is easy to mark.

SAQ 7

Can you think of any disadvantages of multiple choicetesting? Write down as many as you can in the space providedbelow.

Many teachers feel that their pupils need to be taught the skill of answeringmultiple choice questions, especially comprehension questions. However this skill isquite simple, even for text comprehension questions. Teach your pupils to:

look first at the question only, not the options

listen to (or read) the text

try to work out what they think the answer is

look at the options.

4.1.2 Cloze TestsCloze tests are a type of integrative tests. A cloze test consists in a passage

usually taken from an authentic text in which every n thword is deleted. In the classiccloze test there is, therefore, no pre-selection of the blanks, which can cover all types ofwords. In order to fill in the blanks, pupils have to understand the passage as a whole(from the context), and fill in the blanks both in accordance with the meaning and

function of the piece of discourse as a whole and in accordance with the rules ofgrammar at sentence level. In doing this the learner demonstrates an ability to:

read or listen for gist and process the information

predict and reconstruct meaning from the contextual clues surrounding theblanks

show an awareness of the relevant rules of co-occurrence (grammatical,stylistic and lexical).

A cloze test is therefore a truly integrative test. The question is whether it is acommunicative test or not.

Some argue that when pupils encounter an unknown text, there are gaps inmeaning for them represented by the words they do not understand. A cloze testreproduces this situation with the purpose of either assessing how well pupils cope with


13/18

Anca Cehan 13

this, or training them to make informed guesses and predictions based on existingknowledge applied to the contextual clues available.

Another favourable point about cloze tests is that they are easy to construct, bycomparison with multiple-choice tests. All you have to do is pick a suitable text (about250 300 words) and remove every fifth, seventhorninth word (fifth can be too difficult more than tenth is too easy!) Keep the first and the last sentences intact. Alternatively,

you can delete only words by part of speech or content area vocabulary.Here is how you can create a cloze test, starting from an instructional material

given in a textbook in use. The text is taken from High Flyer, Upper Intermediate, byAna Acevedo and Marisol Gower, Longman, 1999, p. 17:

Anya woke up with a start. She knew straight away that today would be aspecial day. She had that familiar feeling in her stomach as if lots of butterflies weretrapped and were flapping their wings, trying to escape.

She had had that feeling just before her grandmother died and when her fatherwas awarded a prize for his research. She felt, more than knew, most things beforeanybody else had heard about them, perhaps even before they had actually happened.

To those who didnt know her well, Anya seemed strange; to those who did know herwell, she was simply special.

That bright summer morning Anya felt restless and the butterflies in herstomach were growing stronger by the minute. She knew from experience that it was nogood fighting it: she would have to follow it.

Mum! Ive got to go out, she announced.London was extremely hot and its streets crowded. Anya walked wherever her

feet took her, as if pulled by some invisible force. And still, the funny feeling in herstomach was intensifying. Eventually, she reached Piccadilly Circus, which was evenbusier than usual.

Shh! Take it easy, she whispered to the feeling in her stomach. Whatever it is,

it wont be long now!He stopped to let the slow traffic advance before crossing the street. A coach

stopped right in front of her. Anya saw herself reflected in the window. Then suddenlyshe realised. What she was looking at was not herself, but a boy, about her age, sittingin the coach. Hey looked at each other with equal surprise and delight.

Anya tried to call out but no sound came out of her mouth. Just then, the coachmoved forward and disappeared in to the London traffic.

Replace the deleted words with blanks that are about the same size andnumber each blank consecutively, e.g.:

She (4) had that feeling just before her grandmother died (5) when herfather was awarded a prize for (6) research. She felt, more than knew, most things (7) anybody else had heard about them, perhaps even (8) they had actuallyhappened. To those who didnt . (9) her well, Anya seemed strange; to those who (10) know her well, she was simply special.

Three levels of difficulty can be constructed from one reading passage bydeleting every fifth, seventh or ninth word. Passages with every fifth word omitted will bemore challenging than those with every ninth word deleted. The greater the number ofblanks, the more reliable the cloze becomes as an indicator of the pupils reading ability.


14/18

Anca Cehan 14

SAQ 8

Could you now mention a few differences between clozetests and gap-filling tests?

One important difference is that a cloze test contains unseen elements, as thegaps are not especially chosen but are every n thword. However, the gaps can also bedistributed according to criteria such as word class, redundancy or predictability.

Although they are very easy to construct, cloze tests are not necessarily veryeasy to do, and they can be frustrating for the pupils if the blanks occur too frequentlyfor them to have sufficient contextual clues, or if the language level of the text is abovethe pupils own productive level.

In cloze tests the relationship between productive and receptive knowledge is a

close one as pupils are being asked to read the passage intensively and complete itgrammatically, so a grasp of the gist is not enough. It can be very demoralising for thepupils to find that all they can do is recognise which part of speech can fit, but notunderstand the passage sufficiently to choose a word.

To score a cloze, count all words that are semantically and syntactically corrector contextually appropriate. Accept any word that is contextually appropriate, not onlyexact word replacements. In this way you can maintain both flexibility and objectivity,because the range of predicted responses is not limitless, since it is controlled by theimmediate and general context.

5 Communicative Testing

How often do our tests measure the pupils communicative ability? We stillprimarily test knowledge of the grammatical and lexical system with some attempt tosee how pupils put this to work in the skills. You will probably have noticed that theexamples given so far are language-based not task-based tests; the emphasis is onwhat pupils know or dont knowrather than what they can or cant dowith the languageat their disposal.

There are several reasons for this apparent lack of relevance of testing toteaching for communication. One of them is that greater use of communicative testingstill awaits the resolution of many issues in communicative materials design andcommunicative methodology.

Another reason for the delay relates specifically to one of the problems of acommunicative view of the language: what constitutes a core syllabus forcommunicative purposes? What functions and notions need to be developed?

As there is some uncertainty of how to test communicative ability in a wayappropriate to the work done in class, we continue to test the language as before. Weknow that in this way we can at least rely on that for information about our pupils graspof the linguistic system. This reveals at least something about communicative ability,especially at lower levels. A lack of knowledge of the basic tenses for instance willseriously impede and limit effective communication. We may even use integrative tests

such as cloze tests to give us information about the pupils ability to process thelanguage. These tests will tell us at what level the pupils can process language andwhatlanguage (broadly speaking) they can process.


15/18

Anca Cehan 15

However, we often find that either the result does not accord with out subjectiveassessment of the pupils overall ability in class or we get a shock when our pupilsappear that they cant usewhat they know to communicate. This poses the problem:what should the criteria for communicative testing be, if existing tests are inadequate asa measure of communicative ability?

Let us now look at some of the features of language use that do not seem to bemeasured in conventional tests.

Interaction-basedIn the vast majority of cases, language in use is based on an interaction. Even

cases such as letter writing can be considered as weak forms of interaction as theyinvolve an addressee, whose expectations will be taken into account by the writer.These expectations will affect both the content of the message and the way in which it isexpressed.

A more characteristic form of interaction is represented by face-to-face oralinteraction that involves not only the adaptation of expression and content but also acombination of receptive and productive skills. What is said by a speaker dependscrucially on what is said tohim.

UnpredictabilityThe development of an interaction is unpredictable. The processing of

unpredictable data in real time is a vital aspect of using language.

ContextAny use of language takes place in a context, and the language forms that are

appropriate vary in accordance with this context. Thus a language user must be able tohandle appropriacy in terms of both context of situation (e.g. physical environment,role/status of participants, attitude/formality) and linguistic context (e.g. textual

cohesion).

PurposeEvery utterance is made for a purpose. Thus a language user must be able to

recognise why a certain remark has been addressed to her/him, and be able to encodeappropriate utterances to achieve her/his own purposes.

PerformanceThe pupils actual use of language.

Authentici ty

The characteristic of language that is not simplified to take account of thelinguistic level of the addressee. An important feature of the pupils ability to useauthentic language is their capacity to come to terms with what is unknown.

Behaviour-basedThe success or failure of an interaction is judged by its participants based on

behavioural outcomes. More emphasis needs to be placed in a communicative contexton the notion of behaviour.

A test of communication must take as its starting point the measurement ofwhat the pupils can actually achieve through language. None of the tests we haveconsidered has set themselves this task.


16/18

Anca Cehan 16

6 Involving Pupils in Handl ing Testing

6.1 Involving Pupils in MarkingHere are a few ideas that you can use so that pupils mark the test themselves.

In this way, you can distinguish between careless mistakes and real misunderstandingsor gaps in their knowledge; pupils also get some individual attention from you.

Here is the procedure for this objective:

1. When a pupil thinks s/he has finished, and brings the test to you, check itwithout ticking or crossing, but merely writing the number of mistakes at the bottom. Thepupil then goes back and tries to find and correct the mistakes.

2. The pupil comes back, and if there are still mistakes, this time you indicateroughly wherethe mistakes are e.g. in that line or No. 4.

3. The pupil comes back again. This time, if necessary, you indicate the typeofmistakes.

4. If you cannot correct the test on the spot, make sure the pupil learns the

correct form through remedial work as soon as possible.

5. Take the tests in to get an overview of what general remedial work isnecessary.

SAQ 9

Could you explain what advantages you see in involvingpupils in marking?

As you look at a progress test, consider:

what level the test is aimed at

what aspects of language are being tested (i.e. grammar, vocabulary,pronunciation, style)

what aspects of skill are being tested, if any

whether functional/communicative ability is being tested

whether each test item tests what it means to test.

6.2 Involving Pupils in Constructing TestsHere are some steps you may want to follow, when your pupils already have

experience of progress tests written by you.

1. Choose a grammatical or lexical area that is not too complicated, or tell yourpupils to look back over work done during a given period.

2. Give them plenty of time in groups to prepare the test and homework time forwriting it up.

3. Give examples of a few techniques they might use.

4. Make sure they have understood that they must limit what they test to whathas been covered in theirclass.


17/18

Anca Cehan 17

5. Supervise the test writing to make sure the tests are correct. In this way thewriting of the test will act both as a learning tool and as further practice.

6. Facilitate the writing by (a) grouping the pupils appropriately (e.g. avoidputting all the weak pupils together), and make sure there is a good mixture in eachgroup; (b) have 3 or 4 groups and give each group a specific area to test.

When you administer the test, you can photocopy neatly written versions forclass or homework. A scoring system must also be worked out.

SAQ 10

What could be, in your opinion, the advantages of suchan approach to testing?

However, problems can occur with students less willing to take responsibility fortheir own learning, and it can be time-consuming. Therefore it is possibly best used withclasses who are used to group work. It is also important to limit what is asked of them totest, either by the size and number of areas, or by the number of questions they haveto produce.

Summary

Most of this unit deals with formal testing. This is done though placement,diagnosis, progress, proficiency, selection, and aptitude tests. You are probably mainlyinterested in the first three, though you may be asked to construct or administerplacement tests, too.

The criteria of test assessment are also discussed: validity, reliability, scorabilityand administrablity.

The testing techniques include discrete items (used in testing grammar,vocabulary, functional exponents, pronunciation, style and include such test items asgap filling), integrative tests (which make use of skills and global awareness oflanguage; examples of integrative tests: cloze, dictation, reading aloud); andcommunicative tests (which test all skills). All these testing techniques have their placein the testing system.

Bear in mind that when testing you need to consider: the pupils' educationalbackground, their language level, the pupils' age and the emphasis of the syllabus youare using: is it structural? functional? skill-based?

Remember that tests are as important to pupils as they are to you. Since theyconstitute a formal measure of progress or current language level, they are verymotivating targets for pupils to work towards. The degree of formality with which youadminister the test will depend on the educational expectations of the pupils and therelationship that you have built up with them.

Key Concepts

formal and informal testing

progress tests


18/18

A C h 18

diagnostic tests

placement tests

assessing tests

reliability

validity

scorability

subjective and objective tests discrete item, integrative and communicative testing

Further Reading

1. Harmer, J., 2001, The Practice of English Language Teaching, Longman, pp.321 - 334 (Testing students)

2. Heaton J. B., Writing English Language Tests,Longman, pp. 159 - 165(Validity and reliability), pp. 171 - 173 (Types of tests)

1

methodology testing

Documents