regional smarter balanced assessment workshops january/february 2014
TRANSCRIPT
Introductions• Jennifer Judkins
• OSPI Mathematics Assessment, EOC• [email protected]
• Anton Jackson • OSPI Mathematics Assessment, Grades 6 – 8• [email protected]
• Julie Wagner • OSPI Mathematics Assessment, Grades 3 – 5• [email protected]
• Kim Andersen• OSPI Collection of Evidence• [email protected]
• Scott Killough• OSPI Collection of Evidence• [email protected]
Housekeeping• Be sure to sign-in.• Be on time when returning from breaks.• Use cell phones and personal electronic devices at
breaks.• Keep side conversations to a minimum.• Use the Parking Lot for non-item-writing questions.• Collect evaluations and provide clock-hour forms at end of
day.
Workshop Program
• Smarter Balanced Assessment System Overview• Shifts in the CCSS• Smarter Balanced Summative Assessment• Claims and the CCSS• Progressions• Smarter Balanced Resources• Claim 1 Item Writing• Claim 2-4 Item Review
Purposes and Goals of Workshop• Increase understanding of the Smarter Balanced
assessment system• Increase understanding of Smarter Balanced claims, item
changes, and how the shifts in the CCSS manifest in items
• Observe how the progression of learning is assessed through the years
• Bring awareness to the Smarter Balanced resources that are available to educators
• Provide participants with knowledge of the item-writing process
5
A Balanced Assessment SystemPage 7
Common Core State
Standards specify
K-12 expectations for college
and career
readiness
All students
leave high
school college
and career ready
Teachers and schools
have information and tools
they need to improve teaching
and learning
Summative: College and career
readiness assessments for
accountability
Interim: Flexible and open
assessments, used for actionable
feedback
Formative resources:
Digital Library with instructional and
professional learning resources for
educators to improve instruction
FORMATIVE RESOURCESDigital Library with formative assessment instructional and professional learning resources for educators
Digital Library Leadership Structures
National Advisory Panel (NAP)
• 11-20 experts• Began December
2012• Provided policies and
criteria for resources
State Leadership Team (SLT)• 5- 8 members• Recruiting began
November 2012• Provides training for
State Network of Educators
State Network of Educators (SNE)
• 70 to 100 members• Representation from
LEAs, AEAs, content leaders, ELL, IHE
• Began recruiting in January 2013
• Provides reviews and resources
9
Resources in the Digital Library
10
• Commissioned Professional Learning Modules• Resources for students and families• Frame the formative assessment process within a Balanced
Assessment System• Articulate the formative assessment process• Highlight formative assessment practices and strategies
Assessment Literacy Modules
• Commissioned Professional Learning Modules• Instructional materials for educators • Instructional materials for students• Demonstrate/support effective implementation of the formative
assessment process• Focus on key content and practice from the Common Core State
Standards for Mathematics and English Language Arts
Exemplar Instructional Modules
• High quality vetted resources and tools for students and families• Reflect and support the formative assessment process• Reflect and support the Common Core State Standards for English
Language Arts/Literacy and Mathematics• Create Professional Learning Communities
Education Resources
* Resources include the following file types: Video, HTML5, Audio, PPT, Excel, Word, and PDF.
Digital Library Functionality
11
Enable trained educators (SNEs and SLTs) to submit, review, and publish resources Allow educators to view, download, and rate resourcesUse state-of-the-art tagging and search to quickly find resources by CCSS and other topicsEnable educators from across the Consortium to collaborate and share their knowledge
Resource Submission Process
SNE1
Submit Resource
withCover Profile
SNE1
SNE
Choose Resource to Review in SNE’s Queue
SNE
Initial Review of Resource with Gate-Keeping Criteria
SNE
Review of Resource with Quality Criteria
RubricReviewed by 3 SNEs
Resource Passes Gate-Keeping
Resource Fails Gate-Keeping—Return to
SNE to Revise
SNE SNE
SNE 1:The SNE who reviews the resource with the Gate-Keeping Criteria will also be one of the
3 SNE reviewers.
Interim AssessmentTwo Types:
Interim Comprehensive Assessments (ICA)• Fixed form in fall 2014 for all grades• Adaptive (as pool allows)• Will be taken online in same form as summative• Some level of hand-scoring
Interim Assessment Blocks (IAB)• Fixed form for most blocks in all grades• Adaptive for certain blocks (as pool allows)• Adaptive for all blocks – later• Will be taken online in same form as summative• Some level of hand-scoring
IABs for math: Current Thinking on CoverageProposed Interim Assessment Blocks
(IAB)Grades 3-5
1 Operations and Algebraic Thinking
2Numbers and Operations in Base Ten
3 Fractions
4 Geometry
5 MeasurementProposed Interim Assessment Blocks
(IAB)Grades 6-7
1Ratios and Proportional Relationships
2 Number System
3 Expressions and Equations
4 Geometry
5 Statistics and Probability
Proposed Interim Assessment Blocks (IAB)
Grade 8
1 Expressions and Equations - I
2 Expressions and Equations - II
3 Functions
4 Geometry
Page 15
IABs for math: Current Thinking on CoverageProposed Interim Assessment Blocks (IAB)
High SchoolAlgebra and Functions
1 Linear Functions2 Quadratics3 Exponentials4 Polynomials5 Radicals6 Rationals7 Trigonometric
Geometry8 Transformations9 Right Triangle Ratios
10 Three-Dimensional Geometry11 Proofs12 Circles13 Applications
Other14 Statistics and Probability
Page 16
SUMMATIVE ASSESSMENT
Assessment Claims
Test Map
Computer Adaptive Testing (CAT)
Item Types
Performance Tasks
Smarter Balanced Claims
• Claims are the broad statements of the assessment system‘s learning outcomes, each of which requires evidence that articulates the types of data/observations that will support interpretations of competence towards achievement of the claims.
Overall Assessment Claims
• Claim for Grades 3-8:
• “Students can demonstrate progress toward college and career readiness in mathematics.”
• Claim for Grade 11:
• “Students can demonstrate college and career readiness in mathematics.”
Assessment Claims 1- 4• Claim 1 – Concepts & Procedures• Claim 2 – Problem Solving• Claim 3 – Communicating Reasoning• Claim 4 – Modeling and Data Analysis
The claims are supported by the Standards for Mathematical Practice.
Major and Supporting/Additional Clusters
• http://www.k12.wa.us/CoreStandards/Mathematics/default.aspx
Test Map Example – Grades 6-8• http://
www.smarterbalanced.org/wordpress/wp-content/uploads/2011/12/Smarter-Balanced-Preliminary-Test-Blueprints.pdf
Estimated Testing Times for 2014 Field Test and 2015 Operational Assessment
Test TypeGrade
s CAT
Perf. Task Only Total
Class Activity Total
English Language
Arts
3-5 1:30 2:00 3:30 :30 4:00
6-8 1:30 2:00 3:30 :30 4:00
11 2:00 2:00 4:00 :30 4:30
Mathematics
3-5 1:30 1:00 2:30 :30 3:00
6-8 2:00 1:00 3:00 :30 3:30
11 2:00 1:30 3:30 :30 4:00
COMBINED
3-5 3:00 3:00 6:00 1:00 7:00
6-8 3:30 3:00 6:30 1:00 7:30
11 4:00 3:30 7:30 1:00 8:30
Times are estimates of test length for most students. Smarter Balanced assessments are designed as untimed tests; some students may need and should be afforded more time than shown in this table.
Computer Adaptive Testing
• “Based on student responses, the computer program adjusts the difficulty of questions throughout the assessment. … By adapting to the student as the assessment is taking place, these assessments present an individually tailored set of questions to each student and can quickly identify which skills students have mastered.”
Smarter Balanced
Summative Assessment Parts• Computer Adaptive Testing (CAT)
• Item Types• Single response multiple choice, Multiple response multiple
choice, Equation/Numeric• Matching Tables, Drag and Drop, Hot Spot, Fill-in Table
• Scoring• Machine Scored
• Performance Tasks• Item Types
• Same as CAT• Text Response
• Scoring• Machine Scored• Hand Scored
Shift One: Focus strongly where the Standards focus
• Significantly narrow the scope of content and deepen how time and energy is spent in the math classroom
• Focus deeply only on what is emphasized in the standards, so that students gain strong foundations
Mathematics topics intended at each grade by at least two-thirds of A+ countries
Mathematics topics intended at each grade by at least two-thirds of 21 U.S. states
The Shape of Math in A+ Countries
1 Schmidt, Houang, & Cogan, “A Coherent Curriculum: The Case of Mathematics.” (2002).
Focusing attention within ‘Number and Operations’
Operations and Algebraic Thinking
Expressions and Equations
Algebra
Number and Operations—Base Ten
The Number System
Number and Operations—Fractions
K 1 2 3 4 5 6 7 8 High School
Shift Two: Coherence Think across grades, and link to major topics within grades
• Carefully connect the learning within and across grades so that students can build new understanding onto foundations built in previous years.
• Begin to count on solid conceptual understanding of core content and build on it. Each standard is not a new event, but an extension of previous learning.
Informing Grades 1-6 Mathematics Standards Development: What Can Be Learned from High-Performing Hong Kong, Singapore, and Korea? American Institutes for Research (2009, p. 13)
4.NF.4. Apply and extend previous understandings of multiplication to multiply a fraction by a whole number.
5.NF.4. Apply and extend previous understandings of multiplication to multiply a fraction or whole number by a fraction.
5.NF.7. Apply and extend previous understandings of division to divide unit fractions by whole numbers and whole numbers by unit fractions.
6.NS. Apply and extend previous understandings of multiplication and division to divide fractions by fractions.
6.NS.1. Interpret and compute quotients of fractions, and solve word problems involving division of fractions by fractions, e.g., by using visual fraction models and equations to represent the problem.
Grade 4
Grade 5
Grade 6
CCSS
Coherence Example: Grade 3
The standards make explicit connections at a single grade
Properties of Operations
Area
Multiplication and Division 3.OA.5
3.MD.7c3.MD.7a
Shift Three: RigorEqual intensity in conceptual understanding, procedural skill/fluency, and application
• The CCSSM require a balance of:• Solid conceptual understanding• Procedural skill and fluency• Application of skills in problem solving situations
• This requires equal intensity in time, activities, and resources in pursuit of all three
Place Value Concepts
• 206 = ____ hundreds ____ tens ____ones• 206 = ____ tens ____ ones• 206 = ____ ones• 347 = ____ tens ____ ones• 452 = 3 hundreds ____ tens ____ones• 521 = ____ hundreds 22 tens ____ ones
Is the comparison true or false?• 12 tens + 11 ones > 1 hundred + 2 tens
(b) Fluency
• The standards require speed and accuracy in calculation.
• Teachers structure class time and/or homework time for students to practice core functions such as solving equations so that they are more able to understand and manipulate more complex concepts.
• Select all of the equations that have integer solutions:
• A) = x - 4• B) x - = • C) - 4 = x• D) = 25• E) 2x =
Fluency Beyond the Facts
(c) Application
• Students can use appropriate concepts and procedures for application even when not prompted to do so
• Provide opportunities at all grade levels for students to apply math concepts in “real world” situations, recognizing this means different things in K-5, 6-8, and HS
• Teachers in content areas outside of math, particularly science, ensure that students are using grade-level-appropriate math to make meaning of and access science content
Operationalizing the CCSS Shifts
Do:
Read the standards including critical areas of focus. Know the structure of the standards.
Know the major, supporting and additional clusters for your grade.
Study the progression documents with colleagues.
Think about unit design first, then lesson, then task. It is not about what you are going to ‘do’ but what the students will learn.
Activity
• Turn to your elbow partner:• Explain in your own words what the three shifts are in the CCSS.
• Explain to your partner how these shifts will change your instructional practice.
Six Key Concepts of Evidence-Centered Design
6. Develop Items or Performance Tasks
1. Define the domain Common Core Standards Math
2. Define claims to be made 4 Math ClaimsContent Specifications
3. Define assessment targets Knowledge, Skills, & Abilities
4. Define evidence required Evidence to be Elicited from Student
5. Develop Task Models Methods for Eliciting Evidence
Universally Designed Items and Tasks – Some Guidelines
• Simplify vocabulary and sentence structure
• Place problems in familiar contexts with familiar items
and names
• Use graphics only when necessary
• Simplify graphics and tables
• Avoid the use of color and maximize contrast
• Avoid content that spans multi-pages or that contains
multiple-elements
• Use highly interactive manipulations only when necessary
• Identify terms and visuals that should not be altered
Bias Concerns
• Bias occurs when content contained in an item or task creates an unfair disadvantage for a sub-group of students• Unfamiliar contexts or examples
• Unusual names of people or places
• References to local events or issues
Jimmy is the star on his football team. This season he scored ten touchdowns, kicked twenty extra points, and had eight field goals.
What is the total number of points Jimmy has scored this season?
What is this question
asking me?
What is a field goal?104
Sensitivity
• Sensitivity focuses on content that creates unease, provokes negative feelings, or challenges beliefs or values
• Religions, religious practices, and religious figures
• Political topics
• Issues of gender, race, and ethnicity
Assessment Claims• Claim 1 – Concepts & Procedures
• “Students can explain and apply mathematical concepts and interpret and carry out mathematical procedures with precision and fluency.”
• Claim 2 – Problem Solving• “Students can solve a range of complex well-posed problems in pure
and applied mathematics, making productive use of knowledge and problem solving strategies.”
• Claim 3 – Communicating Reasoning• “Students can clearly and precisely construct viable arguments to
support their own reasoning and to critique the reasoning of others.”
• Claim 4 – Modeling and Data Analysis• “Students can analyze complex, real-world scenarios and can
construct and use mathematical models to interpret and solve problems.”
Math Practices in the Claims
• Claim 1 – MP 4, 5, 6, 7, 8
• Claim 2 – MP 1, 5, 7, 8
• Claim 3 – MP 3, 6
• Claim 4 – MP 2, 4, 5
Comparing the Claims - Activity
• Individually review the items on the ‘Comparing the Claims’ Worksheet.
• Review with a partner the items by claim number.
What are the differences in the way the problems are written by claim?
How does the focus on four claims change the way you instruct and assess students in the classroom?
CAT Item Types
• Multiple Choice• Single correct answer• Multiple correct answers
• Equation/Numeric• Matching Table• Fill-in Table• Grid Items
• Drag and Drop• Graphing• Hot Spot
Multiple Choice – single correct response
Example Stem 1 (VE): Which model shows 18 of the whole figure shaded?
A.
B.
C.
D.
Grade 3
C)
D)
Multiple Choice – multiple correct responses
A)
B)
Select all the graphs that show a proportional relationship between x and y.
Grade 8
C)
D)
Equation/Numeric
A triangular-shaped garden is shown.
Enter the area of the garden, in square meters.
Grade 5
Drag and Drop
Kayla asked 10 students in her class whether they owned a dog or a cat or both.Drag one number into each box to complete the table, given this information: 40% of the students own a dog.30% of the students own a cat.10% of the students own both a dog and a cat.
Grade 8
Item Type Implications
Review the ‘Item Type’ worksheet and talk to a partner about these questions. Be prepared to share your discussion:
• What implications do these item types have, if any, on your instruction and assessment of students?
• What resources do you have that might allow you to create similar items and imbed them in instruction and assessment?• N-rich• Virtual manipulatives
Performance Tasks• 6 questions
• Distributed among Claim 2, 3, and 4• Item types on CAT may be used in a PT
• Also included short, constructed response:• “Show your work.”• “Explain your reasoning.”
• Classroom activity to precede work on PTs• 2 items automatically scored, rest hand-
scored
Performance Task Implications
Talk to a partner about these questions and be prepared to share your discussion:
• How would you prepare students for sustained, connected activities like the performance tasks?
• What resources do you have for authentic, real-world applications of the mathematics students are learning?• Illustrative Mathematics• Engage NY
Progression Documents • http://ime.math.arizona.edu/progressions/
Progression of Learning and Items through the Grades
Individually review the items in the progressions document.
Find an example where what is learned in a prior year is fundamental to what is learned later.
With a partner, determine the new learning that is added from year to year.
Be prepared to share.
For Educators• Field Test• Practice and Pilot Tests• Sample Items and Performance Tasks• Item Writing and Review
• Content Specifications• Item Specifications• Considerations, Style Guide, etc.
• Achievement Level Descriptors and College Content Readiness
Cognitive Rigor and Depth of Knowledge
• Level of complexity of cognitive demand • Level 1: Recall and Reproduction
• Requires eliciting information such as a fact, definition, term, or a simple procedure, as well as performing a simple algorithm or applying a formula.
• Level 2: Basic Skills and Concepts• Requires the engagement of some mental processing beyond
a recall of information.
• Level 3: Strategic Thinking and Reasoning• Requires reasoning, planning, using evidence, and explanations
of thinking.
• Level 4: Extended Thinking• Requires complex reasoning, planning, developing, and
thinking most likely over an extended period of time.
Level 1 – Recall and ReproductionSimplify each expression.
74
7–3
1
3
2 1
3
9
(–5)6
(–5)10
87 8–12 ·
·
Level 2 – Application of Skills/ Concepts
A cylindrical tank has a height of 10 feet and a radius of 4 feet. Jane fills this tank with water at a rate of 8 cubic feet per minute. How many minutes will it take Jane to completely fill the tank without overflowing at this rate? Round your answer to the nearest minute.
Level 3 – Strategic Thinking andReasoningThe total cost for an order of shirts from a company consists of the cost for each shirt plus a one-time design fee. The cost for each shirt is the same no matter how many shirts are ordered.
The company provides the following examples to customers to help them estimate the total cost for an order of shirts.
• 50 shirts cost $349.50
• 500 shirts cost $2370 Part A: Using the examples provided, what is the cost for each shirt, not including the one-time design fee? Explain how you found your answer. Part B: What is the cost of the one-time design fee? Explain how you found your answer.
Level 4 – Extended Thinking
During the task, the student assumes the role of an
architect who is responsible for designing the best plan for
a park with area and financial restraints.
The student completes tasks in which he/she compares the
costs of different bids, determines what facilities should be
given priority in the park, and then develops a scale
drawing of the best design for the park and an explanation
of the choices made.
This investigation is done in class using a calculator, an
applet to construct the scale drawing, and a spreadsheet.
Depth of Knowledge
• Look at the ‘Comparing the Claims’ and determine the Depth of Knowledge level of each.
• Be prepared to share.
Targets by Grade Level
Grade Level Assessment Targets Documents
3 A - K 114 A - L 125 A - K 116 A - J 107 A - I 98 A - J 1011 A - P 16
Item Specification Overview
• As each section of an item specification document is reviewed, find the same section in an item specification document at your grade-band level.• Grade 3-5 – Target G, Grade 4
• Grade 6-8 – Target A, Grade 6
• Grade 11 – Target M, Grade 11
Statements of Required
Evidence (Derived
from Standards)
Allowable Response
TypesAllowable Tools
Allowable Stimulus
Materials and Mathematics Vocabulary
Task Models Flow from Evidence Statements
94
Details about stimuli
for task model
Required evidence is identified
Task Models Flow from Evidence Statements
95
Stem asks for the required evidence
Stimulus gives you, the item
writer, information
Context should be changed from item
to item
Task Models Flow from Evidence Statements
Use the same stem language
from item to item
Numbers will change from item
to item
Procedures that Support Validity
• Item Specifications documents• Start with a clear explanation of what students are expected to know and be able to do
• Control irrelevant sources of variability by:• Standardizing items using Task Models.• Standardizing items using a Style Guide.
97
Claim 1 Activity
• Choose a task model from your item specification document and write an item to meet that task model.
• Share and review an item with a partner.
• How can you use this process to write items for your classroom or the district?
Claim 2 - 4 Activity
• Go through the “Claims 2 – 4 Sample Items” document. Identify characteristics of these sample items:• What are distinguishing features of items in each claim?• How are the questions asked differently in each claim?• How are claim 2 – 4 items different than Claim 1 items?
• Share your thoughts with a partner.
• Be prepared to share what you have learned about Claim 2 – 4 items.
Targets for Claim 2• Target A: Apply mathematics to solve well-posed
problems in pure mathematics and arising in everyday life, society, and the workplace.
• Target B: Select and use appropriate tools strategically.
• Target C: Interpret results in the context of a situation.
• Target D: Identify important quantities in a practical situation and map their relationships (e.g., using diagrams, two-way tables, graphs, flowcharts, or formulas).
Targets for Claim 3
• Target A: Test propositions or conjectures with specific examples.
• Target B: Construct, autonomously, chains of reasoning that will justify or refute propositions or conjectures.
• Target C: State logical assumptions being used.
• Target D: Use the technique of breaking an argument into cases.
• Target E: Distinguish correct logic or reasoning from that which is flawed and – if there is a flaw in the argument – explain what it is.
• Target F: Base arguments on concrete referents such as objects, drawings, diagrams, and actions.
Targets for Claim 4• Target A: Apply mathematics to solve problems arising in
everyday life, society, and the workplace.
• Target B: Construct, autonomously, chains of reasoning to justify mathematical models used, interpretations made, and solutions proposed for a complex problem.
• Target C: State logical assumptions being used.
• Target D: Interpret results in the context of a situation.
Targets for Claim 4• Target E: Analyze the adequacy of and make
improvements to an existing model or develop a mathematical model of a real phenomenon.
• Target F: Identify important quantities in a practical situation and map their relationships.
• Target G: Identify, analyze, and synthesize relevant external resources to pose or solve problems.
Day 1 Wrap-Up
• What are the next steps you would do to use the information you learned today in your classroom, building and/or district?
• What role will you play in these next steps?
• Evaluations• Clock hours
Day 2 Item Writing Agenda• Welcome Back• Forms and Paperwork• Group Norms• Item-Writing for the Common Core Assessment
• Purposes and Goals• Importance of Validity• Documents Used to Support Validity
• Claim 1 Item Writing Considerations• Procedure for Writing and Reviewing Items• Claim 1 Item Writing• Claim 4 Item Writing Considerations• Claim 4 Item Writing
108
Forms and Paperwork
• Sign - in• Non-Disclosure Agreement
• The items are secure, but the process is not.
• Substitute Reimbursement• Evaluation and Clock Hours
109
Public Disclosure• Please be aware that your participation on this committee is considered public record under Washington State Public Record Act RCW 42.56.
• Please be assured that your personal and residential information are not released.
110
Group Norms• Be on time when returning from breaks.• Use cell phones and personal electronic devices
at breaks.• Keep side conversations to a minimum.• Respect each other’s work styles (need for quiet).• Ask for and give feedback.• Use the Parking Lot for non-item-writing
questions.
111
Purposes and Goals of Workshop• Write assessment items that are aligned to the
Common Core State Standards for Mathematics• Increase understanding of Smarter Balanced item
specifications and other item-writing documents• Involve Washington educators in the assessment
process• Provide participants with knowledge to share item-
writing process with others and increase student achievement
112
Mathematical PracticesMathematical Practices that support Claim 1:
• MP 4: Model with mathematics• MP 5: Use appropriate tools strategically.• MP 6: Attend to precision.• MP 7: Look for and make use of structure.• MP 8: Look for and express regularity in repeated reasoning.
Item Writing Considerations
Reviewed yesterday:• Alignment to CCSS-M Standards• Item Specifications and Task Models• Plain Language and Vocabulary• Bias and Sensitivity• Universal Design• Cognitive Complexity
Moodle PowerPoint:• Style Guide
115
Achievement Quintiles
Five divisions are:• 1 - Very easy• 2 - Easy• 3 - Medium• 4 - Hard• 5 - Very hard
Maximum, Minimum Grade
• Claim 1 items will often be written to a specific grade level so both maximum and minimum grade will be the same. Some items where there is a direct building of skills from grade to grade may make the maximum or minimum grade one above or below.
Score PointsMaximum Points
• Score Points:• 0, 1 for a one-point item• 0, 1, 2, for a two-point item.
• Maximum Points:• 1 for a one-point item• 2 for a two-point item.
Depth of Knowledge• DOK 1—Recall; perform a rote procedure• DOK 2—Application of Skill/Concept• DOK 3—Strategic Thinking• DOK 4—Synthesizing information and
creating new example
• Most of Claim 1 items will be DOK 1 or 2 but watch what is written in the item specification documents.
• If multiple DOKs are listed, you must pick one.
Allowable Response Types
DAS
Multiple Choice, single correct response (MC)
Multiple Choice, multiple correct response (MS)
Matching Tables T/F and Y/N variations (MA)
ITS
Equation/Numeric (EQ)
Fill-in Table (TI)
Drag and Drop, Hot Spot, and Graphing (GI)
Lessons Learned from Certification Set
Successes• Identifying the Task Model• Following the Task Model
• Task models without context must be followed explicitly• Task models with context have a bit more flexibility
• Alternatives for MC and MS items• Rubric and Scoring information
Foci of Attention• Including only what you want the “student” to read in the item• Rationales for MC and MS alternatives
Rationales• MC and MS items need rationales for each
alternative• Use “Correct Answer” for correct answer(s)• Item spec documents usually give examples of reasonable
distractors (incorrect alternatives)• You can include other types of distractors
• Describe, in positive terms, or show work as to why students would choose a distractor:• The student added 3 + 4 before multiplying by 6• The student multiplied the exponents 4 and 7 instead of
adding the exponents• Student adds the leg lengths of the triangle and does not
square the lengths first
Steps to Write Item• Work with a partner.• Choose a standard from the folders and take an item-specific sheet.
• Check item specification for item considerations for the given target.
• Check task model rules for item type, cognitive complexity, required evidence, tools
• Write item.• Follow Peer Review Process.
126
Peer Review Process
• Ask a pair of colleagues to work your item and make suggestions for edits.
• Submit for approval.• Specialist will review item and provide feedback to writers and peer reviewers, if appropriate.
• Specialist may ask for revisions/edits. Writers make edits and resubmit item to same specialist.
127
Claim 4
• Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems.
Modeling and Data Analysis
• Modeling is the process of choosing and using appropriate mathematics and statistics to analyze empirical situations, to understand them better, and to improve decision-making.
Smarter Balanced
Activity• Review the Claim 4 Document with your group.
• Front Matter• Assessment Targets• Sufficient Evidence for Claim 4• Task Models and Example Items
• Be prepared to speak to the differences between writing Claim 1 items and writing Claim 4 items.
Modeling Contexts/Situations Do we
• Brainstorm contexts/situations that are appropriate for your grade band
• Consider specific contexts from the designated standards for this claim
• Contexts/situations can include content from multiple domains.
Claim 4 Mathematical Practices
• Mathematical Practices that support Claim 4:
• MP 2 – Reason abstractly and quantitatively.• MP 4 – Model with mathematics.• MP 5 – Use appropriate tools strategically.
Secondary Claim
• Items written primarily to assess Claim 4 will necessarily involve some Claim 1 content targets.• The content should draw upon the knowledge and skills articulated in the progression of standards leading up to and including the targeted grade
• Related Claim 1 targets should be listed below the Claim 4 targets in the metadata form.
Primary and Secondary Standard/Cluster
While tasks developed to assess Claim 4 will have a primary content focus, components of these tasks will likely produce enough evidence for other content domains that a separate listing of these content domains needs to be included where appropriate.
Targets for Claim 4• Target A (TM1): Apply mathematics to solve problems
arising in everyday life, society, and the workplace.
• Target B (TM 2): Construct, autonomously, chains of reasoning to justify mathematical models used, interpretations made, and solutions proposed for a complex problem.
• Target C (TM3): State logical assumptions being used.
• Target D (TM 4): Interpret results in the context of a situation.
Targets for Claim 4• Target E (TM 5): Analyze the adequacy of and make
improvements to an existing model or develop a mathematical model of a real phenomenon.
• Target F (TM 6): Identify important quantities in a practical situation and map their relationships.
• Target G*: Identify, analyze, and synthesize relevant external resources to pose or solve problems.
* Measured in Performance Tasks only
Assessment Targets• Any given task should provide evidence for several of the
preceding assessment targets; each of the targets should not lead to a separate task. Multiple targets should be listed in order of prominence as related to the task.
Distinguishing Claim 4• Distinguishing between Claim 4 and Claims 1 and 2:• In early grades when equations are still new to students,
an important distinction between Claim 2 and Claim 4 is requiring a model that would lead to a problem's solution.
• In Claim 2, problems are well posed, while in Claim 4, they may have extraneous or missing information.
• In Claims 1 and 2, measurements of objects or figures can be accurately determined. In Claim 4, modeling is used to make approximations.
• In Claim 1, data analysis is straightforward and procedural. In Claim 4, the analysis should be tied to some useful purpose in the real world.
A Review of Metadata Fields• Mathematical Practices• Achievement Quintiles• Maximum, Minimum Grade• Score Points• Maximum Points• Depth of Knowledge• Presentation Formats• Response Type
Scoring
• Scoring rules and answer choices will focus on a student’s ability to use the appropriate reasoning. For some problems, multiple correct responses and/or strategies are possible.
• Items may be worth 1 or 2 points; Justification for more than 1 point must be clear in the scoring rules.
Lessons Learned from Certification Set
Successes• Identifying the Task Model• Following the Task Model
• Task models without context must be followed explicitly• Task models with context have a bit more flexibility
• Alternatives for MC and MS items• Rubric and Scoring information
Foci of Attention• Completing the metadata table• Including only “student” information in item• Rationales for MC and MS alternatives
Preparing to Write Claims 4 Items• Claim 4 document – Standards assessed• Claim 1 document – Development Notes• Context and Standard(s) assessing• Claim 4 document – Targets, Task Models, and Sample
Items• Item Type, Metadata, Scoring, Rubric
Claim 4 Items
• Start with these item types:• MC, MS, and EQ
• Do not copy examples or try to clone them.
• Items must be original and contain several standards that are in alignment with the chart for that grade level.
Item Writing Process• Review Claim 4 document carefully.• Work with a partner.• Decide to which target you will write.• Use the Claim 1 documents as a reference for how the problem is posed to assess Claim 1.
• Identify how the Claim 4 item you are writing differs from Claim 1 items.
Debriefing Claim 4
• How was this writing different from Claim 1?
• The process for Claims 2 and 3 is similar. How might you prepare to write these types of items?
• What are your next steps in your classroom, building, or district?
Day 2 Item Writing Wrap-up• Parking Lot Questions/Comments• Feedback
• Suggestions to improve process.• Other questions?
• Evaluations• Clock hours• Substitute reimbursement form
150