p2p cadre meeting september 29, 2009 welcome! enjoy some breakfast and networking!
TRANSCRIPT
![Page 1: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/1.jpg)
P2P CADRE MEETINGSEPTEMBER 29, 2009
Welcome! Enjoy some breakfast and networking!
![Page 2: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/2.jpg)
Updates
Modules and kit status Teaching timeframe Calendar review for the year Upcoming in-district meetings Today’s meeting
![Page 3: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/3.jpg)
ATLAST ITEM WRITING WORKSHOP
Horizon Research, Inc.May 15-16, 2009
Made possible by a grant from the National Science Foundation (Grant no. EHR-0335328). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the presenters and do not necessarily reflect the views of the National Science Foundation.
![Page 4: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/4.jpg)
Classroom Vignette
After a unit on seasons, a science teacher included the following item on a test:
Which of the following locations would you expect to have the least
temperature variation throughout the year?(a) Toronto, Canada(b) London, England(c) Sydney, Australia(d) Manta, Ecuador
The teacher had done a thorough job teaching the unit, and based on student responses to questions she asked during class, she was confident they would recognize a location on the equator as the correct answer. She was disappointed and confused when three-fourths of the class missed the item. As she returned the tests, she asked several students why they got the question wrong. Almost without exception, each student asked, “Where’s Ecuador?”
Excerpted from: Taylor, M. & Smith, S. (2009). How do you know if they’re getting it? Writing assessment items that reveal
student understanding. Science Scope, 32(5), 60–64.
![Page 5: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/5.jpg)
Meeting Goals
1. Familiarity with a process for developing student assessment items.
2. Familiarity with some principles for developing student assessment items.
3. Opportunity to apply the process and principles for developing student assessment items.
![Page 6: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/6.jpg)
Overview of Day
Overview an assessment development process
Clarifying the content domain Principles for writing student
assessment items Practice applying principles Write items in small groups Revise items in large group Orient to interviewing
![Page 7: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/7.jpg)
A PROCESS FOR DEVELOPING ASSESSMENT ITEMS
![Page 8: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/8.jpg)
Identify Science Topicsand MisconceptionsAbout Those Topics
Write MultipleChoice Items
InterviewStudents andRevise Items
Pilot Items
CreateAssessment
![Page 9: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/9.jpg)
CLARIFYING THE SCIENCE CONTENT TO BE ASSESSED
![Page 10: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/10.jpg)
Assessment Development
![Page 11: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/11.jpg)
Clarifying the Science Content
In ATLAST, we started with middle grades benchmarks.
Flow of matter and energy in living systems
Force and motion Plate tectonics
![Page 12: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/12.jpg)
Overview of Benchmark Clarifications
Within each benchmark, we identified narrowly defined sub-ideas.
Sub-ideas are the smallest assessable piece of information
Within each sub-idea, we identified relevant common student misconceptions.
![Page 13: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/13.jpg)
Why define the content domain?
It affects the validity of the assessment.
The content domain determines the boundaries of the assessable content.
![Page 14: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/14.jpg)
CULLING STUDENT MISCONCEPTIONS RELATED TO THE SCIENCE CONTENT
![Page 15: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/15.jpg)
Why identify common student misconceptions?
They are a source of plausible distractors for assessment items.
![Page 16: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/16.jpg)
Sources of Student Misconceptions
Syntheses of researched misconceptions Content experts and educators with K-12
experience K-12 students
![Page 17: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/17.jpg)
Eliciting Student Ideas
Answers to open-ended questions inform teachers of how students think about the content
Sample prompts for each content area are included in binder
Key questions organize the prompts by the particular aspect of the content they target
![Page 18: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/18.jpg)
SOME PRINCIPLES FOR ITEM WRITING
![Page 19: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/19.jpg)
Some Principles for Item Writing
Content Facilitating comprehension Structure Format Avoiding test-wiseness
![Page 20: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/20.jpg)
CONTENT
![Page 21: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/21.jpg)
Content
1. Target the item; one sub-idea per item if possible.
Why? Precision Good coverage of all sub-ideas within the
benchmark Cognitive load gets higher as more sub-ideas
are being assessed within one item
![Page 22: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/22.jpg)
Content
2. Necessity principle: The knowledge in the sub-idea is needed to answer the item, so the right answer should be about the sub-idea.
![Page 23: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/23.jpg)
The Necessity Principle
The knowledge in the sub-idea is needed to answer the item.
OR
The item cannot be answered correctly without knowing the sub-idea.
![Page 24: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/24.jpg)
Practice Applying the Principle: Items 1A, 1B, and 1C
For each item,… Read the targeted sub-idea, correct
answer, and principle(s) being violated
Review the item
Identify/highlight why the item violates the principle(s)
![Page 25: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/25.jpg)
Item 1A-Flow of Matter and Energy
Which one of the following energy transformations happens in a plant?
A. Light energy is transformed into chemical energy.
B. Heat energy is transformed into chemical energy.
C. Heat energy is transformed into motion energy.
D. Motion energy is transformed into heat energy.
![Page 26: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/26.jpg)
Item 1B-Force and Motion
A student throws a ball straight up. Which one of the following best describes the motion of the ball as it falls down?
A. The ball moves faster and faster because of the force due to gravity.
B. The ball moves slower and slower because the force from the kick is acting against gravity.
C. The ball moves slower and slower because the force of motion is opposing the force due to gravity.
D. The ball moves at a constant speed because gravity is constant.
![Page 27: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/27.jpg)
Item 1C-Plate Tectonics
In which one of the following places would it be most likely for an earthquake to happen?
A. Along the Pacific coast of North America
B. Along the Atlantic coast of North America
C. In the middle of the Pacific Ocean
D. In the middle of North America
![Page 28: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/28.jpg)
Content
3. Sufficiency principle: The knowledge in the benchmark is ALL a student needs to know to answer the item correctly.
The distractors may be about other sub-ideas as long as they’re within the benchmark, but it’s best to have distractors also be in the sub-idea.
![Page 29: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/29.jpg)
The Sufficiency Principle
The knowledge in the benchmark is ALL a student needs to know to answer the item correctly.
OR
Answering the item correctly requires NO knowledge outside the benchmark.
![Page 30: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/30.jpg)
Force and Motion Benchmark
An unbalanced force acting on an object changes its speed or direction of motion or both.
![Page 31: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/31.jpg)
Example:
Which of the following is a force that opposes motion?
A. AccelerationB. InertiaC. MomentumD. Friction
![Page 32: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/32.jpg)
Practice Applying the Principle: Items 2A, 2B, and 2C
For each item,… Read the targeted sub-idea, correct
answer, and principle(s) being violated
Review the item
Identify/highlight why the item violates the principle(s)
![Page 33: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/33.jpg)
Item 2A-Flow of Matter and Energy
Which one of the following is NOT food for humans?
A. SugarsB. FatsC. ProteinsD. Vitamins
![Page 34: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/34.jpg)
Item 2B-Force and Motion
A girl slides a box across the floor. Which of the following is a force that acts on the box?
A. The energy the girl transfers to the box
B. The momentum of the box
C. The inertia of the box
D. The friction between the floor and the box
![Page 35: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/35.jpg)
Item 2C-Plate Tectonics
Why do earthquakes occur more often in some areas than in others?
A. Earthquakes tend to occur where there are large oil deposits underground.
B. Earthquakes tend to occur in desert regions, where the ground is unstable.
C. Earthquakes tend to occur near the edges of plates.
D. Earthquake location is currently not well understood by scientists.
![Page 36: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/36.jpg)
Content
4. Item should not teach content.
![Page 37: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/37.jpg)
Example:
A girl gets on her skateboard and pushes off with her foot. As she moves across the gym floor, she moves slower and slower and eventually comes to a stop. Why does she come to a stop?
A. Because the force due to her foot runs out.B. Because the force in the skateboard runs out.C. Because the force of friction is acting on the skateboard.D. Because the force due to gravity is greater than the force due to friction.
![Page 38: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/38.jpg)
Correction:
A girl gets on her skateboard and pushes off with her foot. As she moves across the gym floor, she moves slower and slower and eventually comes to a stop. Why does she come to a stop?
A. Because the force due to her foot runs out.B. Because the force in the skateboard runs out.C. Because the force of friction is acting on the skateboard.D. Because there are no forces acting on the skateboard.
![Page 39: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/39.jpg)
Practice Applying the Principle: Items 3A, 3B, and 3C
For each item,… Read the targeted sub-idea, correct
answer, and principle(s) being violated
Review the item
Identify/highlight why the item violates the principle(s)
![Page 40: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/40.jpg)
Item 3A-Flow of Matter and Energy
Which one of the following do plants NOT need to make food?
A. WaterB. LightC. Carbon dioxideD. Soil
![Page 41: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/41.jpg)
Item 3B-Force and Motion
A boy slides a salt shaker along a table toward the right. Friction acts to oppose the motion of the salt shaker. As the salt shaker slides, in which direction does the force of friction act on the salt shaker?
A. To the left
B. To the right
C. Upward
D. Downward
![Page 42: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/42.jpg)
Item 3C-Plate Tectonics
Which one of the following best describes how fast Earth’s plates move?
A. A few centimeters a yearB. A few centimeters a monthC. A few meters a yearD. A few meters a month
![Page 43: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/43.jpg)
Content
5. Mirroring: Answer choices should reflect the question being asked.
![Page 44: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/44.jpg)
Example:
How do plants get their food?
A. Plants make food from minerals, water, and sunlight.
B. Plants make food from carbon dioxide, water, and sunlight.
C. Plants get food by absorbing water through their leaves.
D. Plants get food by absorbing nutrients through their roots.
![Page 45: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/45.jpg)
Example:
A sailboat is traveling in the ocean, as shown in the diagram. The wind exerts a force that pushes the boat forward (toward the right side of the page), and the water exerts a weaker force to the left.
How do these forces interact?
A. The forces combine to create a total forward force that is smaller than the force due to the wind.B. The forces combine to make a total forward force that is larger than the force due to the wind.C. The forces cancel each other out.D. The force to the left doesn't matter because the force to the right is stronger.
![Page 46: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/46.jpg)
AVOIDING TEST-WISENESS
![Page 47: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/47.jpg)
Avoiding Test-wiseness
Students tend to…
Choose the longest answer
Choose the most “science-y” answer
Rule out answers with strong qualifiers (e.g., only, never, always)
![Page 48: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/48.jpg)
Practice Applying the Principle: Items 4A, 4B, and 4C
For each item,… Read the targeted sub-idea, correct
answer, and principle(s) being violated
Review the item
Identify/highlight why the item violates the principle(s)
![Page 49: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/49.jpg)
Item 4A-Flow of Matter and Energy
Grass is food for cows. How does the cow use the grass?
A. All the grass leaves the cow’s body after the energy is absorbed.B. All the grass leaves the cow’s body after the water is absorbed.C. The grass is broken down and some of it becomes part of the cow’s body.D. The grass is broken down, but none of it becomes part of the cow’s body.
![Page 50: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/50.jpg)
Item 4B-Force and Motion
A school bus is slowing down as it comes to a stop sign. What is true about the forces acting on it while it is slowing?
A. The forces in the forward direction are greater than the forces in the backward direction.B. The forces in the forward direction are less than the forces in the backward direction.C. The only force acting on the bus is friction.D. There are no forces acting on the bus.
![Page 51: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/51.jpg)
Item 4C-Plate Tectonics
Which one of the following is true about Earth’s plates?
A. Earth has one large plate of solid rock.B. Earth has one large plate that has some solid parts and some liquid parts.C. Earth has a number of plates of mostly solid rock.D. Earth has a number of plates, all of which are liquid.
![Page 52: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/52.jpg)
FACILITATING COMPREHENSION
![Page 53: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/53.jpg)
Facilitating Comprehension
7. Keep sentences brief and straightforward, with a simple phrase structure and no additional clauses.
![Page 54: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/54.jpg)
Example:
The South American plate and the Nazca plate are neighboring plates, with no other plates in between. Scientists say that the South American plate and the Nazca plate are moving toward one another.
Which one of the following best explains what they mean when they say that these two plates are moving toward one another?
![Page 55: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/55.jpg)
Facilitating Comprehension
12. Use present tense and active voice only.
![Page 56: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/56.jpg)
Example:
A shopping cart is pushed so that it moves faster and faster. What is true about the situation?
Correction:
A man is pushing a shopping cart. The cart is moving faster and faster. What must be true about the situation?
![Page 57: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/57.jpg)
Facilitating Comprehension
12. Visuals should facilitate the understanding of what is being asked; don’t include a visual just to include a visual.
No extraneous information Should complement the presented scenario Labels should mirror wording in the item
![Page 58: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/58.jpg)
STRUCTURE
![Page 59: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/59.jpg)
Structure
Restricted to four answer choices
Avoid multiple correct answers (i.e., do not use “Both A and B are correct”)
Avoid using “None of the above”
![Page 60: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/60.jpg)
FORMAT
![Page 61: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/61.jpg)
Practice Applying the Principle: Items 5A, 5B, and 5C
For each item,… Read the targeted sub-idea, correct
answer, and principle(s) being violated
Review the item
Identify/highlight why the item violates the principle(s)
![Page 62: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/62.jpg)
Item 5A-Flow of Matter and Energy
When a baby chick develops inside an egg, the yolk in the egg is its only source of food. As the chick grows, the yolk decreases in size. Why does the yolk decrease in size?
A. Matter is destroyed during the growth of the chick. B. The yolk provides the building blocks the chick is made of. C. The yolk is a source of energy for the chick but not a source of body parts. D. The yolk decreases in size to make room for the growing chick.
![Page 63: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/63.jpg)
Item 5A-Flow of Matter and Energy
When a baby chick develops inside an egg, the yolk in the egg is its only source of food. As the chick grows, the yolk decreases in size. Why does the yolk decrease in size?
A. Matter is destroyed during the growth of the chick. B. The yolk provides the building blocks the chick is made of. C. The yolk is a source of energy for the chick but not a source of body parts. D. The yolk decreases in size to make room for the growing chick.
![Page 64: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/64.jpg)
Item 5B-Force and Motion
A car is moving faster and faster along a straight, level section of road. Which of the following must be true about the forces on the car?
A. The total force in the direction of the car’s motion is greater than the total force in the opposite direction.B. There is an increasing total force in the direction of the car’s motion.C. The total force in the direction of the car’s motion is greater than the car’s force.D. There are no forces in the direction opposite to the car’s motion.
![Page 65: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/65.jpg)
Item 5B-Force and Motion
A car is moving faster and faster along a straight, level section of road. Which of the following must be true about the forces on the car?
A. The total force in the direction of the car’s motion is greater than the total force in the opposite direction.B. There is an increasing total force in the direction of the car’s motion.C. The total force in the direction of the car’s motion is greater than the car’s force.D. There are no forces in the direction opposite to the car’s motion.
![Page 66: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/66.jpg)
Item 5C-Plate Tectonics
Locations 1, 2, and 3 are shown on a map of Iceland to the left. The arrows indicate directions of plate movement.
Where would you expect to find the youngest rock?
A. Location 1B. Location 2C. Location 3D. The rock is the same age at all 3 sites.
![Page 67: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/67.jpg)
Item 5C-Plate Tectonics
Locations 1, 2, and 3 are shown on a map to the left. The arrows indicate directions of plate movement.
Where would you expect to find the youngest rock?
A. Location 1B. Location 2C. Location 3D. The rock is the same age at all 3 sites.
![Page 68: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/68.jpg)
Most Important Rules
1. Target the item; one sub-idea per item if possible.
2. Necessity principle: The knowledge in the sub-idea is needed to answer the item, so the right answer should be about the sub-idea.
3. Sufficiency principle: The knowledge in the benchmark is ALL a student needs to know to answer the item correctly. The distractors may be about other sub-ideas as long as they’re within the benchmark, but it’s best to have distractors also be in the sub-idea.
![Page 69: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/69.jpg)
Taylor, M. & Smith, S. (2009). How do you know if they’re getting it? Writing assessment items that reveal student understanding. Science Scope, 32(5), 60–64.
Science Scope Article
![Page 70: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/70.jpg)
Content-Specific Guidelines
Developed during the process of item writing to address issues that arose in student and/or content expert feedback
Please review prior to writing items
![Page 71: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/71.jpg)
WRITING ITEMS
![Page 72: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/72.jpg)
Assessment Development
![Page 73: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/73.jpg)
Writing Items
Goals and Guidelines Helpful hints Logistics
![Page 74: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/74.jpg)
Goals and Guidelines
Goal: Write 2-3 items per pair/group If you want to work independently for a
portion of the time, please make sure you move toward collaboratively drafting, reacting to, and/or editing items after the first 20 minutes.
Use the item writing checklist to “interview” your own items
![Page 75: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/75.jpg)
“Interview” Your Items
Have you clearly defined the content you are assessing? Do you have a good sense of what is fair game?
Does the item clearly target a specific idea? Do students have to know the content to answer
the item? Could they answer it correctly without knowing the content?
Is the target content sufficient for students to be able to answer the item? Do they need to know anything outside the target content?
![Page 76: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/76.jpg)
“Interview” Your Items
Does the item inform students about the correct answer to other items?
Do the answer choices mirror the wording of the item, or is there a mismatch?
Does the item: keep sentences as brief as possible? use present tense and active voice only? make appropriate use of visuals?
Can students answer the item without knowing the content by using common test-taking strategies?
![Page 77: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/77.jpg)
Helpful Hints
Use existing items as fodder for new items (your own or samples).
Use the “related prior student conceptions” as a source of distractors.
Use textbooks, diagrams, etc. to get ideas for items around specific sub-ideas.
![Page 78: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/78.jpg)
GROUP EDITING OF ITEMS
![Page 79: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/79.jpg)
Attack the item; affirm the writer.
![Page 80: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/80.jpg)
OVERVIEW OF STUDENT COGNITIVE INTERVIEW PROCESS
![Page 81: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/81.jpg)
Assessment Development
![Page 82: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/82.jpg)
Purposes
Cognitive interviews help…
1. Determine the item’s validity2. Determine the item’s clarity3. Determine the item’s plausibility; and4. Identify specific improvements/
revisions to the item
![Page 83: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/83.jpg)
Purposes
1. Validity Does the student think the item is asking
what we intended it to ask? Does the student use the knowledge we
are trying to measure to answer the item? Is the student getting the item
correct/incorrect for the appropriate reason (i.e., understanding/not understanding the knowledge being measured rather than the wording being tricky or because of test wise-ness).
![Page 84: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/84.jpg)
Purposes
2. Clarity• To make sure the item is clear and the
wording and/or graphics are not confusing
3. Plausibility• To determine if the scenario is realistic
4. Improvements• To elicit suggestions for alternative
distractors (e.g., one that may be more realistic or better aligned with a common misconception)
![Page 85: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/85.jpg)
Interview Protocol
![Page 86: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/86.jpg)
Prologue
Purpose of the prologue Helps explain what we are doing; Helps put the student at ease:
Interview is voluntary; Interview is anonymous; Stresses that we aren’t testing him/her, rather
we’re trying to determine if the items are any good.
![Page 87: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/87.jpg)
Background Information
Helps put responses into context; Allows us to internally track:
who has been interviewed; around which items; and when.
![Page 88: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/88.jpg)
Procedure
Ask student to read and think aloud as they read the item and consider the answer choices. Gives you a sense if wording is tricky (e.g.,
s/he re-reads the item several times); and Helps you determine why student is
answering the way s/he is (e.g., real understanding, guessing, test wise-ness).
![Page 89: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/89.jpg)
INTERVIEW: PROCEDURE
![Page 90: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/90.jpg)
Item-Specific Questions
1. Why did you choose that answer? [Validity]
2. What did you think of each of the other answer choices? [Validity]
3. Was there an answer choice you were expecting to see, but did not? [Improvements]
![Page 91: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/91.jpg)
Item-Specific Questions
4. Were there any words or diagrams you did not really understand, or situations that made the question confusing? [Clarity and Plausibility]
5. Is there anything about the question that did not confuse you, but that you think might confuse other middle school students? [Clarity and Plausibility]
![Page 92: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/92.jpg)
INTERVIEW NOTES
![Page 93: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/93.jpg)
Interview Notes
Place to capture student thinking Record of which answer choices were
selected, considered, and/or ruled out Coding of item—Did the student answer
the question correctly or incorrectly for the right or wrong reason? (record after the interview)
![Page 94: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/94.jpg)
STUDENT INTERVIEW
![Page 95: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/95.jpg)
INTERVIEW: FORCE AND MOTION ITEM
![Page 96: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/96.jpg)
Force and Motion Item
Sub-idea E: If an object is slowing down, then there is a net force acting on the object in the direction opposite to the object’s motion.
![Page 97: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/97.jpg)
Force and Motion Item
Tim stretches a rubber band between two posts and uses it to launch a toy car, as seen in the picture. He pulls the car back against the rubber band (as shown in 1), then lets go and the car moves across the floor (as shown in 2 and 3). The car slows down and eventually stops after a few meters.
Why does the car slow down and stop?
![Page 98: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/98.jpg)
Force and Motion Item
Why does the car slow down and stop?
A. The force due to the rubber band gets used up.
B. The car’s force of motion eventually runs out.
C. There is a force acting in the direction opposite to the car’s motion.
D. The car starts to slow down once the backward forces are equal to the forward forces.
![Page 99: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/99.jpg)
Force and Motion Item
The student chose the incorrect answer for the right reason—she did not fully understand the content
Content probing re: friction
Asked about all answer choices
![Page 100: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/100.jpg)
PREPARING FOR STUDENT COGNITIVE INTERVIEWS
![Page 101: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/101.jpg)
Interview Tips
Reassure/reinforce good responding, especially early in interview.
Don’t let the interviewee know what the correct answers are during the interview; you can review items after the interview if student desires.
Remember the main goal of interview is to check validity of item, not to uncover student thinking about the content.
![Page 102: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/102.jpg)
Interview Tips
Are there any answer choices that you can definitely rule out?
Are there any new, unfamiliar, or confusing words in the question or answer choices?
Let’s just talk about each answer choice and what you thought they meant by it.
Let’s say you were taking one of those standardized tests and you just had to pick one answer to bubble in on the answer sheet. Which one would you choose?
I’m curious about what they meant by “X.” What do you think they meant by “X”?
I noticed they used the word “X.” If you had to explain to a younger student what “X” meant, what would you say?
![Page 103: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/103.jpg)
Interview Forms
Includes all three content areas Flow of Matter and Energy 1-Green Flow of Matter and Energy 2-Blue Force and Motion-Purple Plate Tectonics-Tan
When interviewing, … Start with your own items, Then move on to other items in your content area, and Talk about items in the other content areas, as time
allows.
![Page 104: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/104.jpg)
TAKE HOME MESSAGES ON ITEM WRITING
![Page 105: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/105.jpg)
Item Writing
There are many item writing principles; a few are key to writing valid assessments.
1. Clarify the content domain2. Target item3. Necessity 4. Sufficiency
![Page 106: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/106.jpg)
Next Steps in the Assessment Development Process
Identify Science Topicsand MisconceptionsAbout Those Topics
Write MultipleChoice Items
InterviewStudents andRevise Items
Pilot Items
CreateAssessment
![Page 107: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/107.jpg)
![Page 108: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/108.jpg)
Next Steps for P2P Item Writing Write science items for upcoming test Conduct 1 student interview of items by
March Reflect on how student interview
assisted/did not assist with item revision
![Page 109: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/109.jpg)
Reflection
On an index card, write 3 things you learned today 2 things you plan to try 1 question you still have
![Page 110: P2P CADRE MEETING SEPTEMBER 29, 2009 Welcome! Enjoy some breakfast and networking!](https://reader035.vdocument.in/reader035/viewer/2022062519/5697c00b1a28abf838cc8080/html5/thumbnails/110.jpg)
Next Meeting
December 14th
Blue Licks Focus of meeting:
Score student work from PoM unit
Revise PoM units Prepare for
mentee training on units