Download - Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint
![Page 1: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/1.jpg)
Michigan Assessment Consortium
Common Assessment Development Series
Module 6 –The Test Blueprint
![Page 2: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/2.jpg)
Developed and Narrated by
Bruce R. Fay, PhDAssessment Consultant
Wayne RESA
![Page 3: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/3.jpg)
Support
The Michigan Assessment Consortium professional development series in common assessment development is funded in part by the Michigan Association of Intermediate School Administrators in cooperation with …
![Page 4: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/4.jpg)
In Module 6 you will learn about
Test blueprints…what they are and why you need them
The components of a test blueprint Criteria for a good test blueprint Test blueprint example
![Page 5: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/5.jpg)
“If you don't know where you're going, any road will take you there.”
George Harrison (1943 - 2001)
"Any Road", Brainwashed, 2002
![Page 6: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/6.jpg)
Assessment with a Purpose
Educational assessment is not something incidental to teaching and learning. It is an equal partner with curriculum and instruction. It is the critical “3rd leg” through which both students and teachers receive feedback about the effectiveness of the teaching and learning process in achieving desired learning outcomes. Assessment closes the loop.
![Page 7: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/7.jpg)
Purposeful Assessment Requires thoughtful alignment –
ensuring that the items on a test fairly represent the… Intended (curriculum) and actual
(instructional) learning targets Relative importance of those targets Level of cognitive complexity associated
with those targets
![Page 8: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/8.jpg)
Useful Feedback
Also requires tests that are… Reliable (consistent; actually measure
something)
Fair (Free from bias or distortions)
Valid (contextually meaningful or interpretable; can reasonably support the decisions we make based on them)
![Page 9: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/9.jpg)
Test Blueprints, Big Picture
Are a simple but essential tool test developers use to design tests that can meet the preceding requirements
Define the acceptable evidence to infer mastery of the targets
![Page 10: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/10.jpg)
Test Blueprints, Details
Explicitly “map” test items to: Learning Targets Levels of Complexity Importance
![Page 11: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/11.jpg)
Decision-making without data…
is just guessing.
![Page 12: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/12.jpg)
But confidently using unsubstantiated data invalidly…
is even worse.
![Page 13: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/13.jpg)
Learning Targets
GLCEs and HSCEs
![Page 14: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/14.jpg)
Learning Target Details
Structured Hierarchical Framework
![Page 15: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/15.jpg)
Taxonomies
For Cognitive Complexity
![Page 16: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/16.jpg)
Bloom’s Cognitive Domain (Revised 2001)Marzano’s Dimensions of Thinking (1989)Webb’s Depth of Knowledge (1997)
Bloom Creating Evaluating Analyzing Applying Understanding Remembering
Marzano
Evaluating Integrating Generating Analyzing Organizing Gathering
Webb Extended
Thinking Strategic
Thinking Skill/concept
use/application Recall
![Page 17: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/17.jpg)
Putting it all together…A Basic Test Blueprint
Table (matrix) format (spreadsheet) Rows = learning targets (one for each
GLCE, HSCE, etc.) Columns = levels of cognitive
complexity (one column for each level to match the taxonomy you chose to use)
Cells = number of items and points possible
![Page 18: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/18.jpg)
Summary Information
Number of items and points possible:
Row Margins = for that target
Column Margins = for that level of complexity
Lower Right Corner = for the test
![Page 19: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/19.jpg)
Basic Example5 Targets, 3 Levels (DOK)
LearningTargets
Recall# (pts)
Use# (pts)
Strategic# (pts)
Target Totals# (pts)
Target 1 3 (3) 2 (2) 5 (5)
Target 2 1 (1) 2 (2) 2 (4) 5 (7)
Target 3 2 (2) 1 (3) 3 (5)
Target 4 3 (3) 1 (2) 4 (5)
Target 5 2 (4) 1 (4) 3 (8)
Level Totals# (pts)
6 (6) 10 (14) 4 (10) 20 (30)
![Page 20: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/20.jpg)
Is this reasonable?Rule of Thumb Criteria… At least 3 items per target (5 is better) for
reliability Appropriate distribution of items over targets
(1 & 2 appear to be more important than 3 & 5)
Levels of complexity are appropriate for targets and instruction
Appropriate distribution of items over levels of complexity
(all items are NOT at the lowest or highest level)
![Page 21: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/21.jpg)
Limitations…
Shows total points for each target/level combination, but not how those points apply to each item
Doesn’t show item types Doesn’t indicate if partial credit scoring
can/will be used (but may be implied) But…it was easy to construct, is still a
useful blueprint, and is much better than not making one!
![Page 22: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/22.jpg)
Some added sophistication…
It is also useful to keep track of item types / formats to ensure: Appropriate match to learning targets
and associated levels of complexity Balanced use within tests and across
tests over time Track on same or separate spreadsheet
![Page 23: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/23.jpg)
Common item types include…
Selected-response (multiple-choice) (Module 7)
Constructed-response Brief (fill-in-the-blank, short answer, etc.) Extended (outline, essay, etc.) (Module 9)
Performance (Module 8)
![Page 24: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/24.jpg)
Other item types include…
Matching Sort/arrange a list in order Projects
![Page 25: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/25.jpg)
Complexity vs. Utility
Your test blueprint could get complicated if you try to account for too much in one spreadsheet.
Make sure your test blueprint covers the basics, is not a burden to create, and is useful to you
The following example is slightly more sophisticated, but still workable
![Page 26: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/26.jpg)
TargetCode
Item#
Item type SR
Item type CR-B
Item type CR-E
DOKRecallPts
DOKUsePts
DOKStrategicPts
Trgt Tots(pts)
1.1.1.1 1 x 1
1.1.1.1 2 x 1
1.1.1.1 3 x 2 4
1.2.3.4 4 x 3
1.2.3.4 5 x 5 8
etc etc
Col Tots 5 2 2 1 1 7 5 12
![Page 27: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/27.jpg)
Conclusions
Destination (purpose) Road Map (test blueprint) Alignment of items & item types to…
learning targets (curriculum /content) size (complexity) of targets cognitive level of targets relative importance of targets
![Page 28: Michigan Assessment Consortium Common Assessment Development Series Module 6 – The Test Blueprint](https://reader035.vdocument.in/reader035/viewer/2022081517/568154c6550346895dc2cc3a/html5/thumbnails/28.jpg)
Next Module
Modules 1 – 6: Intro and Overview
Modules 7 – 11: The Nitty Gritty
Modules 12 – 18: Making Sure it Works and is Useful