lecture3_testing and evaluation.pdf
TRANSCRIPT
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
1/45
LECTURE 3: TESTING ANDEVALUATION
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
2/45
PRINCIPLES OF LANGUAGE
TESTING AND EVALUATION A test is used to measure the performance of a
student.
Some tests are given under examinationconditions, with invigilators and special
conditions- formal test. Continuous assessment- often made by
teachers all the time in which they are makingjudgements about the pupils performance .
- listening to pupils- observing pupils working individually, in pairsor in groups
- marking their written work
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
3/45
Tests serve many of the needs of teaching,
as we teach we need to monitor progress. Testing and teaching are very closely
interrelated but they do not have the samefocus.
Tests focus on assessing the products oflearning and teaching concentrates onenabling students to succeed in the processof learning.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
4/45
Tests try to assess a students previous learning. A teacher is to ensure that learning takes place
and tests are one of the tools to help a teacher todo so.
A test is only a sample of what the student issuppose to know as it picks out the mostimportant aspects of the skills that have beentaught.
The performance of the given tests is a way ofmeasuring a students mastery of the languageareas he is tested on.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
5/45
A test is often used as a guide on what would
be the most important things to teach and itinfluences what is taught (backwash effect of a
test).
In practice, a test is not always the best guideas it often leaves out important skills because
of practical constraints. For example, the UPSR
& PMR examination do not have the Listening
and Speaking component. Therefore, theteachers do not pay sufficient attention to
these important skills.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
6/45
Some teachers use the formats used in testingfor teaching purposes so as to help students topass examination. This is wrong as what istaught should be decided by reference to how
important the skill is to a students present andfuture life.
A test usually seeks the most economical way ofdetermining what the student already knows.
Therefore leaving out many important contentwhich are used for communication and normallanguage used.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
7/45
Purpose of
Testing A teacher needs a test in order to:
- put students into groups according to their abilities.
- to find out whether a student has learnt what he hasbeen taught.
- to discover if students seem to be able to cope withthe curriculum and to evaluate the effectiveness of
the methodology
- to compare the performance of one student againstthe performance of their peers
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
8/45
Types of Test
Tests fall into 2 categories:- Summative Tests
- Formative Tests
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
9/45
Formative Tests
Formative Tests are used during a
course to monitor the progress of the students. The most common formative tests are
informal. Formal test are also sometimesdesigned.
Type of Formative Test1. Topic/ Progress Tests
- to test how well pupils have learntwhat has been taught
- to test at the end of each topic to be
covered. For example: at the end of eachunit or module to find out if students havemastered what was taught.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
10/45
.....Types of Test: Formative
Purpose of Topic/Progress Tests
- To evaluate effectiveness of teaching and teachingmaterial.
- To obtain information about individual pupils level of
mastery of skills taught.
2. Diagnostic Tests
- attempts to diagnose areas of difficulty
- help teachers in deciding for whom and
what areas of remedial work needs to
be given.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
11/45
.....Types of Test:
Formative
Purpose of Diagnostic Tests
- to obtain information about pupils
strengths and weakness in a specific area.- This information would be an important
input for decisions on material design,provision of remedial classes, etc.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
12/45
.....Types of Test: Summative
Are those that test all that has been taughtin a course.
It usually comes at the end of the course orat the end of each year of the course.
End-of-year examination is a summativetest.
Type of Summative Test
1. Achievement tests- which tests what has been taught
throughout the course/year.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
13/45
.....Types of Test: Summative
Purpose of Achievement Test
- to evaluate effectiveness of teaching andteaching materials and methods.
2. Placement Tests
- are used to help group the studentsaccording to overall entry-level proficiency
of students.- to assess all that a student knows at the
point he begins a new course of study
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
14/45
.Types of Test: Summative
Purpose of Placement Test- to help place pupils into groups
according to ability
3. Proficiency Tests- it is important but not often used in schools.- it is not pegged to any syllabus or course.- it attempts to assess a students future
performance in a specific area of learning orwork.
- it is intended for students from all kind ofbackground
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
15/45
Types of Tests
Norm-Referenced Tests- if the scores are used to compare a studentsperformance with peers.
- a student is considered average when his
results are compared to results of excellentstudents, and appear to be excellent whencompared to another group that is lessproficient.
- good and poor are relative concepts inthis kind of evaluation.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
16/45
Types of Tests
Criterion-Referenced Test
- when a student is required to achieve apre-determined level of proficiency indoing a specific set of tasks, hisperformance is measured not against theperformance of his peers but in terms ofwhether he is able or not to achieve the
stipulated, fixed standard
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
17/45
Types of Tests
Discrete Point Tests
- relate to how this test is constructed
- it aims at testing one and only one point ofgrammar, phonology, vocabulary, or etc. at a time.
Integrative Tests- seek to test global proficiency
- may isolate discrete features like pronunciation,fluency and so on.
- dictation, cloze and all tests based on
communicative tasks
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
18/45
Types of Tests
Communicative Tests
- are based on activities which a personwould recognize as relevant to the use ofthe target language
- giving directions in English is a
communicative test ( would have to do
outside a classroom)
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
19/45
What makes a good test?
Valid
Reliable
Practical
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
20/45
Criteria of A Good Test
A good test must be valid, reliable andpractical.1. Validity
- a test is said to have validity when it tests whatit claims to be testing.
- is concerned with the issue of whether a testmeasures what the tester wants to know aboutthe performance of the target students.
- there are 5 types of validity:i. Face
ii. Contentiii. Predictiveiv. Concurrentv. Construct
( )
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
21/45
Validity summarized by Davies (1968)
Type ofValidity
Test
Face Looks like a good one to learner/layman
Content Accurately reflects the syllabus it isbased on
Predictive Accurately predicts futureperformance
Concurre
nt
Gives similar results to already
validated tests or other immediateexternal criteria (eg: teacherssubjective assessment)
Construct Reflects closely a valid theory offoreign language learning that ittakes as its model
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
22/45
........Criteria of A Good Test
2. Reliability
- is concerned with the precision withwhich the test measures what the
tester wants to know.
- a reliable test must yield the sameresults on the same sample of students
whenever the test is administered and
whoever administers and scores it.
- when several markers are required to mark
candidates performance, reliability is often
low which is caused by inter-variability
i i f d
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
23/45
..Criteria of A Good Test
....Reliability
- Unreliability is minimized by developingperformance bands which describe asaccurately as possible, the language
behaviour at each performance band.
- unreliability is also sometimesdiscovered in the marks awarded by the
same examiner which is called intra-
rater variability.
- it is often caused by factors such as fatique after longhours of marking, long intervals between periods ofmarking, personality factors such as mood of examinerat different periods in the marking, etc
C it i f G d t
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
24/45
........Criteria of A Good Test
3. Practicality
- some test may be ideal in theory,become difficult is practice, eg mass oral testing.
- testing oral proficiency is very important at all stages
in the learning process.- it is very expensive to conduct, very time-
consuming and its reliability is usually rather
low because of inter-rater variability.
A ti it
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
25/45
Activity
Comment on the validity, reliabilty and practicality of the
following tests.1. A group of waiters being assessed while they are working
in a restaurant after they have completed a course on
English for Waiters.
2. A national level essay test.
3. An oral component in monthly tests at school.
4. Reading aloud as a test of oral proficiency
5. National level testing of oral proficiency using languagelaboratories for candidates to record their oral responses
to taped questions
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
26/45
Test Specifications
What to test?
Why you need to test? How you want to test?
You need to spell out the details of your test:
- what kind of sub-tests and how many- what formats you want to use, etc
- what skills will be tested (eg: reading,
writing/ listening)- what sub-skills will be focused on
T t S ifi ti
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
27/45
Test Specifications
Section
SkillSub-skills/ Elements
A Reading Guessing meanings of words usingcontextual clues;
Distinguishing main point fromsupporting ideas;
inference
B Grammar Conjunction
Past Tense
Formal and Informal ExpressionsC Vocabulary Words related to animals,
occupations and family
D Writing Descriptions of people
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
28/45
Writing Test Specifications
Tasks
- need to be carefully chosen to give agood indication of the skills you want
to test.
- For example, to test on speaking
spontaneously, you cannot give a
reading aloud test. You need to
prepare a test : oral interview/
discussion situation- Typical tasks answering comprehension
questions, writing guided compositions, filling
in blanks, etc
W iti T t S ifi ti
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
29/45
Writing Test Specifications
2. Types of Text
- The KBSR and KBSM spell out a number of types oftexts: letters, stories, advertisements, labels,poems,etc.
- In specifying texts, you would need to specify length,eg read a story of 250 words; write a letter of 100
words.- Specify level of difficulty- it is difficult to specify levelof difficulty. One way that testers use is to specify aknown level, for eg the same level of texts as found in
the KBSR school textbook.- Number of texts also need to be specified, forexample, one text of 200 words/ 2 texts, each about300-350 words.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
30/45
Writing Test Specifications
3. Topics
- select the topics for your test from
syllabus specifications;
= monthly test : achievement/progress
test
- you can also choose any topic within
the maturational level of students. For
example, you can choose a text on
People for a Year 4 students but not on
life insurance
W iti T t S ifi ti
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
31/45
Writing Test Specifications
4. Format
- Same skill can usually be tested in more than one way.- For example: reading proficiency can be tested usingMCQ questions, T/F questions, cloze passage, open-ended
questions, etc.
- The format that you would be using in testing should be
specified.
- The number of items in each sub-test need to be spelt
out.- The format should be familiar to the students. If not, use
and introduce in the teaching first.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
32/45
Writing Test Specifications
Component Format No. ofItems
Reading Rational Cloze
Open-ended
15 blanks
5questions
Grammar Cloze
ErrorIdentification
10 blanks
15 items
W iti T t S ifi ti
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
33/45
Writing Test Specifications
5. Weightage
- All the skills in a test are equally important, butnot often.
- So, you need to allocate different marks todifferent sections of your test/ different items in
the same section.- Allocate marks according to the importance ofeach section/item in relation to the test as a wholeand what you want to know from the test results..
- The weightage of each item/section need to bespecified.
Writing Test Spe ifi ations
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
34/45
Writing Test Specifications
6. Time Allocation
- How much time will be given for each sub-test willalso need to be specified.
- 2 things to bear in mind:
(i) how important the skill is
(generally, the more important skillshould be given more time);
(ii) how long the task would take (eg: writing
a free composition would take a longertime than a guided composition/ filling in
blanks in already written composition)
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
35/45
Types of Test Items
1. Close-ended and open ended test items
- it is called according to how much freedom the
testee has in deciding on the scope and quality of
the answer.- a close-ended item has a predetermined answer or
set of answers which are usually short.
- open-ended items- an essay test, an oral interview,a reading or listening comprehension question that
expects students to answer using their own words
f
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
36/45
Types of Test Items
Close-ended items are of 2 types:
1. objective
2. structured
Objective items- option to choose the correct answer from the
alternatives provided (MCQ)
- pure cloze only one answer is accepted
Structured Items
- the structure of the answer is determined by theexaminer. For example: guided composition
Activity 2
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
37/45
Activity 2State whether the items on the test are
objective, structured or open-ended.
1. Fill in the blanks in each of the followingsentences.
Ali jumped .......... The river.
2. Write an composition on the topic , My Mother.
3. Combine the following sentences using the word in brackets.
4. Read the following text and answer the comprehensionquestions that follow:
a. The cat was weak and thin. (True or False)
b. Do you think the cat in the story is well-cared for?
Give reasons for your answer.
c. We know the cat was hungry from the fact that it ...
Types of Test Items
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
38/45
Types of Test Items
Subjective and Objective Tests
- All tests are subjective as it involves sampling.- Sampling means selection
- Selection is usually based on personal preference and
personal judgement.- The method of scoring distinguishes objective from
subjective test.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
39/45
Types of Test Items
Subjective Test
- Each examiner uses his own judgments in evaluating
performance and awarding marks.
- Inter-rater as well as intra-rater variability are probable. Objective Tests
- All items have a predetermined answer or set of answers.
- Whoever marks this section, same person or different, theresult will be same.
Types of Test Items Subjective &
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
40/45
Types of Test Items.... Subjective &
Objective Tests
Objective Test
Strengths- Easy to mark.
- Can be used for mass testing because computermarking is possible.
- High reliability- Item banking for recycling is possible
Weaknesses
- Difficult to construct.
- Low validity
- Sometimes skills and areas tested because theyare testable not because they are important
Types of Test Items Subjective &
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
41/45
Types of Test Items.... Subjective &
Objective Tests
Subjective Test
Strengths
- Easy to set
- High validity
- Can assess affective and interpretiveaspects of language skills
Weaknesses
- Marking is time-consuming
- Reliability is low
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
42/45
Please refer to Nesamalar Chitravelus ELTMethodology Principle & Practice
TOPIC : Testing and Evaluation
Page 330 349 for your further
reference and preparation for your
Tutorial & Practical Presentation.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
43/45
TUTORIAL
In pairs, analyse test items in sample UPSRexam papers.
Practical
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
44/45
Practical
Teaching and Assessing Listening and Speaking
Skills
Workshop: Testing and Evaluation Using UPSR exam papers as samples, prepare
test specifications.
Design the following to assess listening and
speaking skills:- Multiple Choice Questions
- Blank Filling
- Text Completion
- Structured Questions
Present the workshop materials on testing andevaluation.
-
7/27/2019 Lecture3_TESTING AND EVALUATION.pdf
45/45
THANK YOU