course outcomes.com_hour_2_table_of_specifications

1. In April 2011, The CPE PhD, LLC became Course Outcomes, LLC. The training materials are referred to in this packet are owned and copyrighted by Course Outcomes, LLC and can only be used for free trainings. They may not be rebranded nor sold without express written permission from Course Outcomes.
Course Outcomes and The CPE PhD
2011 Course Outcomes, www.courseoutcomes.com

2. Thank you for using The CPE PhDs training services. The content of this training is copyrighted by The CPE PhD and may not be resold, repackaged, or printed for profit.Organization Members may download and repackage materials for their organization but not for profit without express written permission from The CPE PhD.
Intro Slide
3. Managing Directors Message
An introduction to hour 2: Table of Specifications
or test blueprint
4. Table of Specifications
and Test Blueprints
Part II
5. Test Construction Philosophy
600 possible items from my course.
My 60 item test.
Suppose you wrote all possible questions for everything your course has taught a student (600 items or so). If you put all 600 of these items in a box, and pulled out 60 items, this would be a sample taken from your population of 600 items.
A test is a sample of items you could have asked.
Most of us do not have time to write 600 items, nor would we leave our certification tests up to a random drawing, so we create a structure to test the more important elements of our course.
600 possible items from my course.
My 60 item test.
I dont have time to write 600 items so Im going to write 6 items on history of SOX, 4 items on section 2, etc.
We also use intuitive reasoning to tell us that if a person can answer one item, they could probably answer another item.(e.g. If a person correctly knows the square root of 81 and 36, then how likely is it that they know the square root of 25?)
Exactly how many items we need to write to make a generalization that a student knows the content can be answered intuitively via your experience or verified by a psychometrician.
Psychometrician is a fancy way of saying professional test writer.
8. The way a student judges if your test is fair is by three criteria:
1) Is your item sample a fair representation of what they should or could have learned?
Is your test aligned with your instruction?
That test had nothing to do with what we learned!
2) Is a score on the 60 item test generalizable to the larger pool of what the student has learned in the course?
Is your test generalizable?
This test score doesnt really show all that I have learned.
3) Do the 60 items represent (in a fairly weighted fashion) all possible items you could have asked or did you over/under emphasize some points?
Is your test fairly weighted?
My tests only focus on my favorite parts of my course.
Have you ever had a professor like this?
11. Purpose of this Training
Construct a fair test that is aligned with your instruction.
Once your generalinstructional objectives and specific learning outcomes are established you are now prepared to create what is called a Table of Specifications.
12. The table of specifications serves as guide for both instruction and assessment.
Table of Specifications
13. The table of specifications is a table or matrix that has three parts:
A list of content areas
A list of student abilities
Number of Test items
Both the content area and student abilities are derived from your SLOs.
Table of Specifications
14. GIO Students will understand the background of Sarbanes-Oxley legislation.
SLO Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation.
As an Example
15. SLO Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation.
19. SLO Students will be able to identifyspecific dates and events surrounding the Sarbanes-Oxley legislation.
20. Now we are ready to begin planning how many
items should be written.
Notice that you can write several items in each cell that assess students ability to do something with the content.
On what date was the Sarbanes-Oxley legislation passed?
June 24, 2001
October 23, 2003
January 3, 2005
December 4, 2008
What event spurred the passing of the Sarbanes-Oxley legislation?
Exxon Valdees Disaster
Enron Scandal
Other plausible but incorrect option
Other plausible but incorrect option
21. An example of
a table of specifications
22. 2011 Course Outcomes, www.courseoutcomes.com
23. As an example, maybe with audit committees and independent auditors students only need to identify certain guidelines but will never need to implement any procedures regarding these topics.
Notice also that with the exception of the cells with zeros, no cell has less than 2 items.
The numbers represent the number of test items written for each specific learning outcome.
Note that some cells have zeros
This would be done if you plan on having more than one assessment covering the same content.
(e.g.assessments for each day as well as a post-assessment)
Otherwise you run the risk of the students seeing the same item on two different exams.
This will happen when content does not lend itself to a particular student ability.
24. When selecting items for a test it is important to ensure that the test items reflect the emphasis of your instruction.
Alignment of instruction
and assessment
25. For example if hypothetically you spend 90% of your time teaching about financial disclosures
and assessment
26. For example if hypothetically you spend 90% of your time teaching about financial disclosuresand 10% of your time teaching about internal controls,
and assessment
27. For example, if hypothetically you spend 90% of your time teaching about audit controls and 10% of your time teaching accounting statistics, you would want this ratio to be reflected in the test questions.
and assessment
28. 90% of your instruction teaching Y
90% of your
test assessing Y
29. With this principle in mind an actual table of specifications may look like this:
and assessment
31. Now you will be able to see what percentage of the content and student abilities are sampled by your test.
33. At this point you can ask yourself, Do these percentages reflect the emphasis of my instruction and course assignments?
These percentages are calculated
by all of the items in the table; in this case 24.
2/24 = 8%
This means that 8% of the test covers specific dates and events.
It also means that 38% of the assessmenttests students ability to identify
If not you may need tomake adjustments to your -
by dividing the number of items in this entire row
Test
Instruction
and/or
Specific Learning Outcomes
34. Your Assignment
Create a table of specifications for your course which reflects:
(1) a balance of instruction and assessment (i.e. 25% of my instruction focuses on achieving this SLO so my test will measure 25% on this SLO).
(2) the number of items which should be written.
Ensure that your table of specifications has rows for the content, columns for student abilities, and numbers in the cells representing the number of items you will write.
The content and student abilities will be taken from the specific learning outcomes you prepared for the first assignment.
35. CPE PhD test creation tool
42. If you have any questions or need assistance check the discussion board or contact Dr. Daniel Winder at the [email protected]
www.courseoutcomes.com

course outcomes.com_hour_2_table_of_specifications

Education

course outcomes

fair test

specific learning outcomes

item test generalizable

sample of items

test score doesnt

professional test writer

sarbanesoxley legislation