Developing Assessment InstrumentsBy Gerral Pinkston

o b j e c t i v e s .

• Describe the purpose for criterion- referenced tests. .

• Describe how entry skills tests, pretests, practice tests, and posttests are used by instructional designers. .

• Name four categories of criteria for developing criterion-referenced tests and list several considerations within each criterion category. .

• Given a variety of objectives, write criterion- referenced, objective- style test items that meet quality criteria in all four categories. .

• .

Four Types of Criterion-Referenced Tests and Their Uses• Entry Skills Tests

• Pretests

• Practice Tests

• Posttests

Entry Skills Tests

• The first type of test, an entry skills test, is given to learners be-fore they begin instruction. These criterion- referenced tests assess learners’ mastery of prerequisite skills, or skills that learners must have already mastered before be-ginning instruction.

Pretests

• The purpose of a pretest is not necessarily to show a gain in learning after instruction by comparison with a posttest but rather to profile the learners with re-gard to the instructional analysis. The pretest is administered to learners before they begin instruction for the sake of efficiency— to determine whether they have previ-ously mastered some or all of the skills that are to be included in the instruction. If all the skills have been mastered, then the instruction is not needed.

Practice Tests

• The purpose for practice tests is to provide active learner participa-tion during instruction. Practice tests enable learners to rehearse new knowledge and skills and to judge for themselves their level of understanding and skill. Instructors use students’ responses to practice tests to provide corrective feedback and to moni-tor the pace of instruction.

Posttests

• Posttests Posttests are administered following instruction, and they are parallel to pretests, except they do not include items on entry skills. Similar to the pretest, the posttest measures objectives included in the instruction. As for all the tests that are described here, the designer should be able to link the skill ( or skills) being tested with its corresponding item on the posttest.

Test Item Criteria

• Regardless of the type of learning involved in the objective, appropriate test item writ-ing techniques should be applied to the development of criterion- referenced tests. There are four categories of test item qualities that should be considered during the creation of test items and assessment tasks.

Test Item Criteria

• Goal- Centered Criteria

• Learner- Centered Criteria

• Context- Centered Criteria

• Assessment- Centered Criteria

Goal- Centered Criteria

• Test items and tasks should be congruent with the termi-naland performance objectives. They should match the behavior, including the ac-tion and concepts, prescribed. To match the response required in a test item to the behavior specified in the objective, the designer should consider the learning task or verb prescribed in the objective. Objectives that ask the student to state or define, per-form with guidance, or perform independently will all require a different format for ques-tionsand responses.

Learner- Centered Criteria

• Test items and assessment tasks must be tailored to the characteristics and needs of the learners, including considerations such as learner needs, vocabulary and language levels, developmental levels for setting appropriate task complexity, motivational and interest levels, experiences and backgrounds, spe-cial needs, and freedom from bias

Context- Centered Criteria

• In creating test items and assessment tasks, designers must consider the eventual performance setting as well as the learning or classroom environment. Test items and tasks must be as realistic or authentic to the actual per-formance setting as possible. This criterion helps to ensure transfer of the knowledge and skills from the learning to the performance environment.

Assessment- Centered Criteria

• Assessment- Centered Criteria Learners can be nervous during assessment, and well- constructed, professional- looking items and assessment tasks can make the as-sessment more palatable to them. Test- writing qualities include correct grammar, spelling, and punctuation as well as clearly written and parsimonious directions, re-source materials, and questions.

Mastery Criteria

Objective Tests

• Writing Objective Test Items

• Sequencing Items

• Writing Directions

Objective Tests

• Objective tests include test items that are easy for learners to complete and designers to score. The answers are short, they are typically scored as correct or incorrect, and judging correctness of an answer is straightforward. Objective formats include com-pletion, short answer, true/ false, matching, and multiple- choice. Test items that should be scored using a checklist or rubric, including essay items, are not considered to be objective items, and they are described in the next section on alternative assessments.

Writing Objective Test Items

• Whether centered on goals, learners, contexts, or as-sessments, designers can use all four main criteria in developing effective objective test items. These criteria were described in detail previously, and they are presented in the rubric at the end of the chapter for your convenience.

Sequencing Items

• There are no hard and fast rules that guide the order of item placement on a test of intellectual skills or verbal information, but there are suggestions that can guide placement. Final decisions are usually based on the specific testing situation and the performance to be tested.

Writing Directions

• Tests should include clear, concise directions. Beginning a test usually causes anxiety among learners, who will be judged according to their performance on the test. There should be no doubt in their minds about what they are to do to perform correctly on the test. There are usually introductory directions to an entire test and subsection directions when the item format changes.

Chapter summary

• In order to develop criterion- referenced tests, you will need the list of performance objectives based on the in-structionalanalysis. The conditions, behavior, and cri-teria contained in each objective will help you determine the best format for your assessment instrument.

• An objective test format will be best for many ver-balinformation and intellectual skill objectives; how-ever, you still must decide what objective- style item format would be most congruent with the prescribed conditions and behaviors.

• Some intellectual skills cannot be measured using objective test items. Examples include writing a para-graph, making a persuasive speech, and analyzing and contrasting certain features of two different methods for predicting economic trends.