test construction 1
DESCRIPTION
TRANSCRIPT
Are you the kind of Are you the kind of teacher who ask the teacher who ask the following questions?following questions?
Are you the kind of Are you the kind of teacher who ask the teacher who ask the following questions?following questions?
The poem “The Raven” ______
a. was written by Edgar Allan Poeb. was written by Elizabeth
Browningc. was written by Omar Khayyand. was written by Jose Garcia Villa
•Is it NOT true that Magellan discovered the
Philippines?
When did the People Power Revolution take in
the Philippines?A. February 23, 1986B. after the Snap ElectionC. March 1, 1956D. after Valentines’ Day in 1986
•Who was the author of the book quoted in the footnote of Chapter 1 of the present textbook
If you answered “If you answered “YESYES” ” to any of the choices to any of the choices presented then you presented then you
have a have a BIG BIG PROBLEMPROBLEM!!
If you answered “If you answered “YESYES” ” to any of the choices to any of the choices presented then you presented then you
have a have a BIG BIG PROBLEMPROBLEM!!
YOUR STUDENTS ARE NOT LEARNING!!!!
“13% of students who fail in class are caused by faulty test questions”
WORLDWATCHThe Philadelphia Trumpet
August 2005
•It is estimated that 90% of all test questions asked in the US are of “Low level” - knowledge and comprehension (Wilen, W.W., 1992)
• “Low level” doesn’t mean easy:• Write an essay explaining the decline
and fall of the Roman Empire incorporating at least five of the seven causes discussed in class from the writings of Gibbon and Toynbee
• “High level” doesn’t mean hard:• Which movie did you like more, WALL-E
or Cars? Why?
TEST CONSTRUCTION
Presented by:Arnel O. Rivera
http://www.slideshare.net/ArnelSSI
Outline:Part I Principles in Test
Construction Steps in Preparing
Test Questions Preparing Multiple
Choice Questions Preparing True or
False Questions
Part II Review of Part I Preparing Matching
Type Questions Preparing Sentence
Completion Questions Preparing Essay
Questions Other types of Test
Questions Wrap-up/Things to
Remember
“The evaluation of pupils’ progress is a major aspect of the teacher’s job.“
Evaluating Educational Outcomes
(Oriondo & Antonio)
Explain the message of the comic strip.
The Purpose of Testing
To provide a record for assigning grades.
To provide a learning experience for students.
To motivate students to learn.
To serve as a guide for further study.
The Purpose of Testing
To assess how well students are achieving the stated goals of the lesson.
To provide the instructor with an opportunity to reinforce the stated objectives and highlight what is important for students to remember.
Characteristics of Good Tests
Validity – the extent to which the test measures what it intends to measure
Reliability – the consistency with which a test measures what it is supposed to measure
Usability – the test can be administered with ease, clarity and uniformity
Scorability – easy to score
Interpretability – test results can be properly interpreted and is a major basis in making sound educational decisions
Economical – the test can be reused without compromising the validity and reliability
Other Things to Consider
“To be able to prepare a good test, one has to have a mastery of the subject matter, knowledge of the pupils to be tested, skill in verbal expression and the use of the different test format”
Evaluating Educational Outcomes(Oriondo & Antonio)
1. Multiple Choice
2. True or False
3. Matching Type
4. Fill-in the blanks (Sentence Completion)
5. Essay
5 Most Commonly used Test Format
Source: Turn-out of Test Questions in SSI (2003-2007)
General Steps in Test Construction
OUTLINE
DRAFT
ORDER
TEST ANALYZE
SUBMISSION
PRODUCE A T.O.S.
OUTLINE:– the unit learning objectives or
– the unit content or major concepts to be covered by the test
Back to Main Menu
Table of Specifications (TOS)
• A two way chart that relates the learning outcomes to the course content
• It enables the teacher to prepare a test containing a representative sample of student behavior in each of the areas tested.
• Don’t make it overly detailed.
• It's best to identify major ideas and skills rather than specific details.
• Use a cognitive taxonomy that is most appropriate to your discipline, including non-specific skills like communication skills or graphic skills or computational skills if such are important to your evaluation of the answer.
Tips in Preparing the Table of Specifications
(TOS)
• Weigh the appropriateness of the distribution of checks against the students' level, the importance of the test, the amount of time available.
• MATCH the question level appropriate to the level of thinking skills
Tips in Preparing the Table of Specifications
(TOS)
Examples of Student Activities and Verbs for Bloom’s Cognitive Levels
Table 2.1 in Jacobs & Chase (1992:19)
Bloom’s Cognitive Level
Student Activity Words to Use in Item Stem
Knowledge Remembering facts, terms, concepts, definitions, principles
Define, list, state, identify, label, name, who?, when?, where?, what?
Comprehension Explaining/interpreting the meaning of material
Explain, predict, interpret, infer, summarize, convert, translate, account for, give example, paraphrase
Application Using a concept or principle to solve a problem
Apply, solve, show, make use of, modify, demonstrate, compute
Examples of Student Activities and Verbs for Bloom’s Cognitive Levels
Table 2.1 in Jacobs & Chase (1992:19)
Bloom’s Cognitive Level
Student Activity Words to Use in Item Stem
Analysis Breaking material down into its component parts to see interrelationships/ hierarchy of ideas
Differentiate, compare/contrast, distinguish ____from ____, how does ____relate to ___, why does ____work
Synthesis Producing something new or original from component parts
Design, construct, develop, formulate, imagine, create, change, write a poem or short story
Evaluation Making a judgment based on a pre-established set of criteria
Appraise, evaluate, justify, judge, which would be better?
Tips in Preparing the Table of Specifications
(TOS)• The following array shows the most
common questions types used at various cognitive levels.
Factual Knowledge
Application Analysis and Evaluation
Multiple ChoiceTrue/FalseMatching TypeS. CompletionShort Answer/RRT
Multiple ChoiceShort AnswerProblemsEssay
Multiple Choice
Essay
Activity:
• Prepare a short TOS using the selection in your activity sheet.
Back to Main Menu
DRAFT the questions covering the content in the
outline
Back to Main Menu
ORDER the selected questions logically.
Place simpler items at the beginning to ease students into the exam.
Group item types together under common instructions.
If desirable, order the questions logically from a content standpoint (e.g. chronologically or by conceptual groups, etc.)
Back to Main Menu
Test• PUT the questions away for one or
two days before rereading them or have someone else review them for clarity.
• TEST the questions by actually taking the test.
Back to Main Menu
•ANALYZE the items to give you an idea whether the questions were well-written or poorly written as well as if there were problems in understanding instruction.
Back to Main Menu
General Rules in Writing Test Questions
Number test questions continuously.Keep your test question in each test group uniform.Make your layout presentable.Do not put too many test questions in one test group. T or F: 10 – 15 questions Multiple Choice: max. of 30 questions Matching type: 5 questions per test group Others: 5 – 10 questions
Some additional guidelines to consider when writing items are described below:
1. Avoid humorous items. Classroom testing is very important and humorous items may cause students to either not take the exam seriously or become confused or anxious.
2. Items should measure only the construct of interest, not one’s knowledge of the item context.
3. Write items to measure what students know, not what they do not know. (Cohen & Wallack)
Multiple Choice Test
When checking the stems for correctness:
Ensure that the stem asks a clear question.
Reading level is appropriate to the students
The stem is grammatically correct.
Negatively stated stems are discouraged.
What to Look for on Multiple Choice Tests
Example:
What is the effect of releasing a ball in positive gravity?
a) It will fall “down.” correct
b) It will retain its mass. true but unrelated
c) It will rise. false but related
d) Its shape will change. false and unrelated
What to Look for on Multiple Choice Tests
Multiple Choice Questions1. Use negatively stated stems sparingly and
when using negatives such as NOT, underline or bold the print.
2. Use none of the above and all of the above sparingly, and when you do use them, don't always make them the right answer.
3. Only one option should be correct or clearly best.
Multiple Choice Questions:
4. All options should be homogenous and nearly equal in length.
5. The stem (question) should contain only one main idea.
6. Keep all options either singular or plural.
7. Have four or five responses per stem (question).
Multiple Choice Questions:
7. When using incomplete statements place the blank space at the end of the stem versus the beginning.
8. When possible organize the responses.9. Reduce wordiness.10.When writing distracters, think of
incorrect responses that students might make.
Examples1. Sheldon developed a highly controversial
theory of personality based on body type and temperament of the individual. Which of the following is a criticism of Sheldon's work?a. He was influenced too much by the Freudian psychoanalysis.b. His rating of physique and temperament were not independent.c. He failed to use empirical approach.d. His research sample was improperly selected.
Examples
Better: (Eliminate excessive wording and irrelevant information)
1. Which of the following is a criticism of Sheldon's theory of personality?
Examples
1. The receptors for the vestibular sense are locateda. in the fovea.b. in the brain.c. in the middle ear.d. in the inner ear.
Examples
Better: (Include in the stem any word(s) that might otherwise be repeated in each option.)
1. The receptors for the vestibular senses are located in the _______.a. foveab. brainc. middle eard. inner ear
Examples
1. Which is not a major technique for studying brain function?a. Accident and injuryb. Cutting and removingc. Electrical stimulationd. Direct phrenology
Examples
Better: (Use negatively stated stems sparingly. When used, underline and/or capitalize the negative word.)
1. Which is NOT a major technique for studying brain function?
Examples
4. ________________ is the least form of behavior disorder.a. Psychosisb. Panic disorderc. Neurastheniad. Neurosis
Examples
Better: (When using incomplete statements avoid beginning with the blank space.)
1. The least severe form of behavior disorder is __________________.
Examples
1. The number of photoreceptors in the retina of each human is abouta. 115 millionb. 5 millionc. 65 milliond. 35 billion
Examples
Better: (When possible, present alternatives in some logical order.)
1. The number of photo receptors in the retina of each human is abouta. 5 millionb. 35 millionc. 65 milliond. 115 million
Examples6. Latane and Darley's smoke-filled
room experiment suggested that people are less likely to help in groups than alone, because peoplea. in groups talk to one another.b. who are alone are more attentive.c. in groups do not display pluralistic
ignorance.d. in groups allow others to define the situation as a non-emergency
ExamplesBetter: (All alternatives should be
approximately equal in length.)6. Latane and Darley's smoke-filled room
experiment suggested that people are less likely to help in groups than alone, because people in groupsa. talk to one anotherb. are less attentive than people who are
alonec. do not display pluralistic ignoranced. allow other to define non-emergencies
Activity:
• Prepare twotwo multiple choice questionsmultiple choice questions based on the selection in your activity sheet.
True or False
Each statement is clearly true or clearly false.Trivial details should not make a statement false.Statements are written concisely without more elaboration than necessary.Statements are NOT quoted exactly from text.
What to Look for onTrue/False Tests
Give emphasis on the use of quantitative terms than qualitative terms.Avoid using of specific determiners which usually gives a clue to the answer. False = all, always, never, every, none,
only True = generally, sometimes, usually,
maybe, oftenDiscourage the use of negative statements.Whenever a controversial statement is used, the authority should be quoted.Discourage the use of pattern for answers.
Tips in Making True/False Tests
Examples:
____ 1. Repetition always strengthens the tendency for a response to occur.
(Using "always" usually means the answer is false.)
Find the errors, and/or problems with the following true-false tests.
Examples:
_____ 2. The process of extinction is seldom immediate but extends over a number of trials.
(Words like "seldom" usually indicate a true statement.)
Examples:_____ 3. The mean, median, and mode are
measures of central tendency, whereas the standard deviation and range are measures of variability.
(Express a single idea in each statement.) e.g.“The mean and standard deviation are measures of central tendency.”
Activity:
• Prepare two true or false questions based on the selection in your activity sheet.