Classroom Assessment: Concepts and Applications

Chapter 6: Creating

Achievement Tests

In Chapter 6 We Will Study:

• Selection and Supply Test Items

• Higher-Level Questions

• Guidelines for Writing and Critiquing Test Items

Achievement Test The most commonly

used procedure for gathering formal evidence about student learning.

A good assessment plan takes many things into

consideration.1st: identifying important instructional

objectives2nd: selecting question formats that match

the objectives3rd: deciding whether to construct one’s

own test or use one from a textbook4th: providing good instruction5th: providing a review and information

about the test

What can go wrong???

• Test items can be poorly constructed

• Test can be objectively scored

The result? Students don’t have a fair chance

to show what they have learnedInformation is not provided for

valid decision making

Tests are composed of short communications called

questions or items.Test items must be …•Brief

•Set clear problems for students to think about

•Complete and independent of other questions

•Be stated in clear, precise language

•Linked to the educational objectives

Selection and Supply Test Items

Please turn to page 146. Selection Items

• Selection items are those in which a student selects a correct answer from among a number of options presented.

• Selection Items Include:

1. Multiple choice

2. True-False

3. Matching

1/12

Selection: Multiple-Choice Items

•These consist of a stem, which presents the problem or question to the student, and•A set of options, or choices, from which the student selects an answer.•May be used to assess higher-level thinking, but primarily used to assess factual knowledge and comprehension levels.

Stem

Options

2/12

Multiple Choice: Advantages/Disadvanta

gesAdvantages:

–Easy to score–Several items can be completed by students in a short period of time

Disadvantages:–Doesn’t allow students to construct, organize, and present their own answers–Susceptible to guessing

3/12

Selection: True-False Items•Requires students to classify a statement into one of two categories, such as true or false; yes or no.

•Primarily used to assess factual knowledge and comprehension.

•May be used to assess higher-level thinking.

4/12

TF

True-False:Advantages/

DisadvantagesAdvantages:

–Easy to score–Several items can be completed by students in a short period of time

Disadvantages:–Doesn’t allow students to construct, organize, and present their own answers–Susceptibility to guessing

5/12

Selection: Matching Items

•Consist of a column of premises, a column of responses, and directions for matching the two.

•The same set of options or responses is used for all the premises.

6/12

Matching:Advantages/

DisadvantagesAdvantages:

–Easy to score–Decreases the amount of reading students must perform in order to display knowledge of several terms, people, or facts

Disadvantages:–Limited mainly to assessing lower-level behaviors

7/12

Supply ItemsPlease turn to page 148.

•Supply items are those in which the student supplies or constructs his or her own answer.

•Includes:–Short Answer–Completion (Fill-in-the-blank)–Essay

8/12

Short-Answer & Completion Items

Short answer: Presents the problem with a direct question.What is the name of the first president of the United States?

Completion:Presents the problem as an incomplete sentence with blanks to fill in.The name of the first president of the United States is __________.

In each case, the student must supply his or her own answer, typically a

word, phrase, number, or sentence.

9/12

Short Answer & Completion:

Advantages/Disadvantages

Advantages:–Fairly easy to construct–Diminishes the likelihood that students will guess answers

Disadvantages:–Tends to mainly assess factual knowledge or comprehension.

10/12

Essay Items:Advantages

Essay questions give students the greatest opportunity to construct their own responses.

Most useful for testing higher-level thinking skills such as analyzing, synthesizing, and evaluating.

Also the primary way teachers assess students’ ability to organize, express, and defend ideas.

11/12

Essay Items: Disadvantages

•Time consuming to answer and score.

•Permit testing only of a limited amount of students’ learning.

•Tend to favor students with good writing ability.

12/12

Higher-Level Questions

•Growing emphasis on teaching and assessing students’ higher-level thinking

•Solving problems, interpreting charts, explaining something in one’s own words, identifying relationships, and carrying out specific activities

Interpretive Exercises

•A common form of multiple-choice item that can assess higher-level thinking.

•Gives students some information or data and then asks students to interpret, comprehend, analyze, apply, or synthesize it.

Graphs, charts, reading passages, pictures, or tables.

Interpretive Exercises: Disadvantages

•Cannot show how students organize their ideas when solving a problem.

•Difficult to construct

•Heavy reliance on students’ reading ability

Interpretive Exercises: Should meet 5 general guidelines

A. Relevance: The exercise should be related to the instruction provided to students. If it is not, it should not be used.B. Similarity: The material presented in the exercise should be new to the students, but similar to material presented during instruction.C. Brevity: There should be sufficient information for students to answer the questions, but the exercises should not become tests of reading speed and accuracy.D. Answers not provided: The correct answers should not be found directly in the material presented. Interpretation, application, analysis, and comprehension should be needed to determine correct answers.E. Multiple questions: Each interpretive exercise should include more than one question to make most efficient use of time.

3/8

Guidelines for Writing & Critiquing Test Items

To improve tests, test items should…1. Cover important objectives emphasized in instruction2. Be written clearly and simply: Seven Rules3. Be reviewed for misleading statements, confusing formatting, or excess verbiage before testing

Rule 1: Write Clearly & Simply:

Avoid ambiguous and confusing wording and structure.

Students are prevented from figuring out what they are being asked and can’t

demonstrate their learning.

Rule 2: Write Clearly & Simply:

Use appropriate vocabulary.

If students can’t understand the vocabulary used in test questions,

their test scores will reflect their vocabulary deficiencies rather than

how much they learned from instruction.

Rule 3: Write Clearly & Simply:

Keep questions short and to the point.

Test items should quickly focus students on the question being asked.

Rule 4: Write Clearly & Simply:

Page 162

Write items that have one correct answer.

With the exception of essay questions, most paper-and-pencil test items are designed to

have students select or supply one best answer.

Rule 5: Write Clearly & Simply:

Give information about the nature of the desired answer.

Essay questions should focus students’ answers on the major points

covered by instruction.Inform students about the nature and

scope of the expected answer.

Rule 6: Write Clearly & Simply:Page 164

Do not provide clues to the correct answer.

Test item writers should take care not to provide grammatical clues, implausible

option clues, or specific determiner clues.

Rule 7: Write Clearly & Simply

Don’t overcomplicate.

Avoid using numbers or words that overcomplicate a given problem. This

increases the likelihood of errors and does not accurately assess the students’ ability.

Review Items Before Testing

•Reread test items.•Match each item to an objective. •Modify or delete items that don’t link to a lesson objective.

• Ask a colleague, etc. to review the items critically.

• Edit test based on feedback.

Each item has at least one fault.Read each item, identify the fault(s) in it and rewrite

the item to correct the fault.

1. Robert Fulton, who was born in Scotland and came to the US in 1843, is best known for his invention of the steamboat that he called the Tom Thumb. True False

32

2. Minor differences among organisms of the same kind are known asa. Heredityb. Variationsc. Adaptationd. Natural selection

Rule 1: Confusing wording

Rule 6: Clue provided

Each item has at least one fault.Read each item, identify the fault(s) in it and rewrite

the item to correct the fault.

3. The recall of factual information can best be assessed with a __________ item.a.matchingb. objectivec. essayd. short-answer

33

4. F. Scott Fitzgerald wrote _________________________.

Rule 6: Clue Provided

Rule 4: Many correct answers.

Each item has at least one fault.Read each item, identify the fault(s) in it and rewrite

the item to correct the fault.

5. Although the experimental research completed, particularly that by Hansmocker, must be considered too equivocal and the assumptions viewed as too restrictive, most testing experts would recommend that the easiest method of significantly improving paper-and-pencil achievement test reliability would be toa. increase the size of the groupb. increase the weighting of the itemsc. increase the number of itemsd. increase the amount of testing time.

34

Rule 1: Confusing; Rule : Inappropriate vocabulary;Rule 3: Not kept short and to the pointRule 7: Over complicated

Each item has at least one fault.Read each item, identify the fault(s) in it and rewrite

the item to correct the fault.

6. Boston is the most important city in the Northeast. T F

35

7. An electric transformer can be useda. for storing up electricity.b. to increase the voltage of alternating current. (correct answer)c. it converts electrical energy into direct current.d. alternating current is changed to direct current.

Rule 1: Confusing wording (undefined: most important)

Rule 1: Confusing sentence structure

Chapter OLC Review

Visit Chapter 6 of the text website for chapter quizzes, related websites, and

other helpful study materials.

www.mhhe.com/airasian6e

Chapter 6: Creating Achievement Tests Name_________________

1. What are the differences between selection and supply items?

2. What are the advantages and disadvantages of selection items?

3. What are the advantages and disadvantages of supply items?

4. What are the differences between higher- and lower-level test items?

5. What are examples of clues to be avoided in multiple-choice, true-false, completion, and matching items?

6. What is an interpretive exercise and why is it a useful method for assessing higher-level thinking?

7. Three guidelines for constructing test questions are:1. Cover important topics2. Write clearly and simply3. Review items before testing.How does each of these guidelines lead to improved test questions?

37

Review of Chapter 6: The Frayer Model

• The Frayer Model (Frayer, Frederick, and Klaumeier, 1969) helps students learn the meanings of key concepts through word categorization.

• Using this method, students form an understanding of concepts by learning

the attributes, choosing examples, and choosing non-examples of the concept.

• Students can use the Frayer Model before, during, or after reading to learn more about a topic.

• Utilizing this instructional model extends student comprehension of the topic and aids in retention of the information.

DefinitionDefinition CharacteristicsCharacteristics

ExamplesExamples Non-examplesNon-examplesWordWord

DefinitionDefinition CharacteristicsCharacteristics

ExamplesExamples Non-examplesNon-examples

•Form 6 Pairs

•Pairs count off

•Construct a Frayer Model for your assigned concept.

1.Multiple-Choice Items2.True-False Items3.Short-Answer Items4.Completion Items5.Essay Items6.Interpretive Exercises