1 writing test blueprints and test items “software for creating and delivering assessments with...
TRANSCRIPT
![Page 1: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/1.jpg)
1
Writing Test Blueprints and Test Items
“Software for Creating and Delivering
Assessments With Powerful Reports”
![Page 2: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/2.jpg)
2
Objectives
204/20/23
Develop test items
Survey the test blueprint
04/20/23Copyright 2000 QuestionMark. All rights reserved.
At the end of this presentation, you will understand how to:
Document the test blueprint
![Page 3: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/3.jpg)
3
Best Practices for Developing Tests
04/20/23Copyright 2000 QuestionMark. All rights reserved.
1.Analyze the job 2.Document the test blueprint 3.Survey the blueprint 4.Develop items 5.Alpha test and revise items6.Beta test the exam 7.Select items and set the cut-score 8.Go live with the test
![Page 4: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/4.jpg)
4
Define objectives in behavioral terms Action verb Condition(s) Standard(s)
Use concrete verbiage Meet key criteria
Sample Work Process for Writing Test Objectives
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 5: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/5.jpg)
5
Major test objectives or topics (job tasks)
Subtopics (subtasks) within each major performance area
Rank test objectives and topics (tasks) in order of importance
How Do You Document the Test Blueprint?
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 6: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/6.jpg)
6
Select contributors Re-prioritize tasks (objectives) Query time spent
How Do You Survey the Test Blueprint?
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 7: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/7.jpg)
7
Question Writing Process
Start from your surveyed blueprint Select a level of difficulty guideline Write question stems Write correct answers Write distractors Edit
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 8: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/8.jpg)
8
What Does a Level of Difficulty Guideline Give You?
Consistency Set performance standards Yardstick to gauge level of difficulty
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 9: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/9.jpg)
9
Level of Difficulty Guideline
Evaluation Synthesis Analysis Application Comprehensio
n Knowledge
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 10: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/10.jpg)
10
Knowledge and Comprehension
Evaluation SynthesisAnalysisApplication Comprehension - explain, predict,
paraphrase Knowledge - define, label, list
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 11: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/11.jpg)
11
Application and Analysis
Evaluation Synthesis Analysis – compare, contrast, relate Application – apply, solve, computeComprehensionKnowledge
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 12: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/12.jpg)
12
Synthesis and Evaluation
Evaluation – appraise, evaluate, judge Synthesis – formulate, construct, create AnalysisApplicationComprehensionKnowledge
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 13: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/13.jpg)
13
Apply Appropriate Test Standards
Formal standards Administrative policies Item writing guidelines
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 14: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/14.jpg)
14
Question (Response) Types
Perception offers eight question types: Multiple choice True/false Multiple response Numeric item Word response Selection item Matrix item Hot spot item
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 15: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/15.jpg)
15
Preferred Item Types
Easily understood Minimize guessing Avoid testing trivial information Avoid candidate scrolling
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 16: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/16.jpg)
16
How to Write Stems
Present a single, definitive problem Use a question format Objective congruence Appropriate level of difficulty Active voice
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 17: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/17.jpg)
17
What to Avoid in Stems
Questions that build Negatively phrased questions “Always” and “Never” “Corporate-centric” questions Humor Trick questions “Trainer-centric” questions
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 18: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/18.jpg)
18
Guidelines to Write Options
Comparable in content, length, & logic Non-repeating Parallel language Logically or numerically ordered Avoid “All of the above” and “None of the
above” Plausible distractors
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 19: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/19.jpg)
19
Avoid Grammatical Clues
04/20/23Copyright 2000 QuestionMark. All rights reserved.
Option A is the only option that makes grammatical sense.
The best way to increase the reliability of a test is to:
A. Increase the test length. [Correct]B. Removing poor quality items. C. Test should be readable for all test
takers.
![Page 20: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/20.jpg)
20
Avoid Association Clues
04/20/23Copyright 2000 QuestionMark. All rights reserved.
Option B includes the word “seven.”
Who are the “Magnificent Seven?”
A. A pro basketball team [Correct]B. A group of seven fictional, heroic,
Western figures [Correct]C. A protest group made famous during
the 1960sD. A vegetarian rock group
![Page 21: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/21.jpg)
21
What is the cost of an item that normally sells for 9.99 USD that is discounted 25%?
Order Options Numerically
04/20/23Copyright 2000 QuestionMark. All rights reserved.
WrongA. 5.00B. 7.50C. 2.50D. 6.66
RightA. 2.50 B. 5.00C. 6.66D.7.50
![Page 22: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/22.jpg)
22
What are the three important concerns in achievement testing? V=validity R=reliability E=efficiency O=objectivity
Order Options Logically
04/20/23Copyright 2000 QuestionMark. All rights reserved.
WrongA. V, R, E B. O, V, RC. R, O, E D. E, O, V
RightA. V, R, O B. V, R, E C. V, E, O D.R, E, O
![Page 23: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/23.jpg)
23
Objective statement by item
Item naming conventions Item difficulty or weight
Evolving the Test Blueprint
04/20/23Copyright 2000 QuestionMark. All rights reserved.
![Page 24: 1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed05503460f94bde9cb/html5/thumbnails/24.jpg)
24
Thank you for your attention
Thank you for your time and attention. We hope you have enjoyed learning about how to write test blueprints and test items.
We at Question Mark strive to improve our products and services. To let us know how you felt about this presentation, visit our North American home page and complete the Learning Center Survey: http://www.questionmark.com/surveys.htm
Question Mark provides numerous presentations to help our customers, and prospective customers, successfully deploy computerized assessments. We provide public “web-seminars” that use a combination of conference calls and web technology to discuss computerized assessment issues. You can register for a public seminar by going to: www.questionmark.com/seminars/index.asp
For other material that you might find useful, please visit our Learning Center at: www.questionmark.com/learningcenter/
04/20/23Copyright 2000 QuestionMark. All rights reserved.