Office of Standards, Assessments and Research & Evaluation
Colorado Assessment Revision
Formative & Interim Subcommittee
August 14, 2010Denver, Colorado
Office of Standards, Assessments and Research & Evaluation
Agenda• Introductions and Goals for the Day• Meeting Norms• Plenary Session and Meeting 1 Review• Colorado Standards• Webinar Debrief: Discussion with Stanley
Rabinowitz• Exemplars• Assessment FOR Learning• Using the Template• Homework
Office of Standards, Assessments and Research & Evaluation
Introductions
Remind Us –
• Who You Are• Where You Are From
Office of Standards, Assessments and Research & Evaluation
Goals for Today• Analyze and discuss assessment research and
exemplars from national and international designs and extract options to consider for recommendation.
• DELIVERABLE: Reach consensus on the purpose of each assessment.*
• DELIVERABLE: Compile an initial draft of concepts for each assessment and prioritize.*
*These are first drafts and can be revised through meeting 4.
Office of Standards, Assessments and Research & Evaluation
Meeting Norms• Participating in and contributing to the
conversation is expected by ALL members.• Use your perspective and expertise to
“expand the pie.”• Listen to others to seek understanding.• Be open minded to new ideas.• Focus on the task and stay on track.• Disagree productively, not personally.• Check your perceptions with the group.
Office of Standards, Assessments and Research & Evaluation
Plenary Session Review
Questions
Concerns
Comments
Office of Standards, Assessments and Research & Evaluation
Meeting 1 Review
Questions and Topics to Revisit• Webinar and Purpose of the Assessments
will be discussed as part of this agenda• Definitions
Interim and Benchmark Assessments: How are they different?
Formative: Can we agree that formative assessment is a process, not a product?
• Other?
Office of Standards, Assessments and Research & Evaluation
Colorado Academic Standards
• Mastery-based• 21st Century Skills and Readiness
Competencies• Vertical and Horizontal
alignment
Office of Standards, Assessments and Research & Evaluation
Essential Components of the Academic Standards
My StandardsMy Future
Office of Standards, Assessments and Research & Evaluation
21st Century Skills• Critical Thinking and Reasoning
(cause and effect, analysis, logic)
• Information Literacy (knowledge acquisition, source discernment, system
management)
• Collaboration (synergy, team resourcing, leadership for new knowledge)
• Self-direction (persistence, adaptability, work ethic, initiative)
• Invention (creativity, innovation)
Office of Standards, Assessments and Research & Evaluation
Content Area: MathematicsStandard: 1. Number Sense, Properties, and OperationsPrepared Graduates: Are fluent with basic numerical and symbolic facts and algorithms, and are able to select and use appropriate
(mental math, paper and pencil, and technology) methods based on an understanding of their efficiency, precision, and transparency
Grade Level Expectation: Eighth GradeConcepts and skills students master:
2. Formulate, represent, and use algorithms with rational numbers flexibly, accurately, and efficiently
Evidence Outcomes 21st Century Skills and Readiness CompetenciesStudents can:a. Add, subtract, multiply and divide rational numbers
including integers, positive and negative fractions and decimals
b. Apply computational methods to solve multi-step application problems involving percents and rational numbers
c. Analyze how credit and debt impact personal financial goals (PFL)
Inquiry Questions: 1. How do operations with rational numbers compare to
operations with whole numbers?2. How do you know if a computational strategy is sensible?3. Why would estimation be used in problem-solving?Relevance and Application:1. Computational fluency with rational numbers allows
individuals to accomplish daily tasks in life and work such as adjusting recipes, comparing the cost of credit from different providers, calculating overtime pay, determining selling prices to make profits, calculating interest, finding percent error, gratuities, or fees.
2. Rational numbers are used extensively in measurement tasks such as home remodeling, clothes alterations, graphic design, and engineering.
Nature of Mathematics:1. Mathematicians describe their processes and solutions
using careful vocabulary and precise notation.
Office of Standards, Assessments and Research & Evaluation
1st GradePredictions and Patterns
DanceUse traveling patterns
in space and time Visual ArtsRecognize
characteristics and expressive features of art and design in
works of art
Theatre Arts and Drama
Create animal and human characters in specific environments through
improvisation
Social StudiesArrange life events
in chronological order
ScienceMake predictions
about how a material on Earth might be useful
based on its properties
PE / HealthPatterned, rhythmic
movement activates the brain to facilitate
learning.
MusicPlay simple patterns
MathExtend a repeating pattern based on a
rule
LiteracyFollow and
replicate patterns in predictable poems
World Language
Use words derived from cognates, prefixes, and
thematic vocabulary
Office of Standards, Assessments and Research & Evaluation
Colorado Academic Standards
• What we are measuring Standards are end of level, mastery-based.
Learning progressions have to be developed. What is the level of granularity that these assessments need to represent?
How do we ensure that the interim assessment is not mini-CSAP?
Are we considering professional development, item/task banks, other?
Office of Standards, Assessments and Research & Evaluation
LUNCH
Office of Standards, Assessments and Research & Evaluation
Webinar DebriefImpressions/Questions:• Formative has substantial impact• Do we have to implement statewide, or can we pilot
recommendations across a few districts?• F/I is a dynamic process• Frustration: disconnect between theoretical and practice• Teaching “how” to take a test vs. academic instruction• 163 and its implications to this work: must include interim
measures – interim strategies first, then focus on formative• 163 supports the role of assessment being more aligned than in
the past to support instructional practice and student achievement• Be clear about what we would like to see happen vs. what the
state role should be• Concern: the recommendations become a “canned” vendor
assessment
Office of Standards, Assessments and Research & Evaluation
Conversation with Stanley Rabinowitz
What is the purpose of interim and formative assessments in a
comprehensive assessment system?
Guiding Questions for Consideration
• What are the state values that the most comprehensive assessment system will represent? What components do we value most?
• What are the milestones for implementation of the various elements of this system? When, how, and by whom should the new components be developed?
• What resources will support the development and implementation of the various measures in this system? How can we supplement these resources to ensure adequate short- and long-term funding for key elements?
• How will we collect feedback from stakeholders and evaluate consequences that may emerge from implementation of the measures in this system?
17
Office of Standards, Assessments and Research & Evaluation
Exemplars
• Explore each exemplar with a partner. • What did you find interesting about each
exemplar? • What are its noticeable deficiencies?• Report your observations with the group
These exemplars are meant to expand your thinking about what is possible in the Colorado
Assessment System.
Office of Standards, Assessments and Research & Evaluation
Exemplars
Florida Assessments for Instruction in Readinghttp://www.fcrr.org/FAIR/index.shtm
Alaska Formative Assessment Resourceshttp://www.eed.state.ak.us/tls/assessment/FormativeAssessmentResources.html
New Zealand: Embedding Formative Assessment In Multiple Initiatives
http://www.oecd.org/dataoecd/53/2/34260418.pdf
Office of Standards, Assessments and Research & Evaluation
Exemplars
The Silicon Valley Mathematics Initiative: Formative Performance Assessment Program
http://www.noycefdn.org/documents/math/formativeassessmentprogram.pdf
Literacy by Designhttp://hmheducation.com/litbydesign/features_7.php
Louisiana Department of Education: EAGLE Systemhttp://www.doe.state.la.us/lde/saa/2273.html
Office of Standards, Assessments and Research & Evaluation
Exemplars
Visible Thinking - Lemshaga Academi, Stockholm. Project Zero, Harvard University
http://www.pz.harvard.edu/vt/VisibleThinking_html_files/VisibleThinking1.html Smarter Balanced Assessment Consortiumhttp://www.k12.wa.us/smarter/
The Partnership for Assessment of Readiness for College and Careers (PARCC)
http://www.fldoe.org/parcc/
Office of Standards, Assessments and Research & Evaluation
Using AssessmentFOR
Learning
Richard Stiggins video
“Classroom Assessment for Student Learning – Doing It Right, Using It
Well”
Office of Standards, Assessments and Research & Evaluation
BREAK
Office of Standards, Assessments and Research & Evaluation
Colorado Assessment Revision Subcommittee Working Template
Office of Standards, Assessments and Research & Evaluation
Template’s First Page Type of Assessment
Summative, Formative/Interim, Postsecondary and Workforce Readiness, School Readiness, and Special Populations
Purpose of the Assessment Utility
What the assessment will and will not measure
Data and Results Postsecondary and Workforce Readiness 21st Century Skills
critical thinking and reasoning, information literacy, collaboration, self-direction, and invention
Office of Standards, Assessments and Research & Evaluation
Template’s Second Page Assessment Guidelines
May include, but may not be limited to, format, mode of delivery, frequency, duration, etc.
Provide rationale for each recommendation Prioritize
Roles and responsibility CDE, CDHE, Districts, Institutes of Higher Education, Schools,
Teachers, Parents, Students
Assessment Resources What resources are needed for successful administration? How might those needs be addressed?
Office of Standards, Assessments and Research & Evaluation
Template’s Third Page Consequences
Positive consequences and the entities or individuals affected Negative consequences and the entities or individuals
affected with possible solutions
Barriers Identify barriers and recommendations to overcome them
Alternative Considerations List other ideas discussed by the subcommittee and provide
the rationale for their exclusion
Office of Standards, Assessments and Research & Evaluation
Template’s Fourth Page
References, Resources and Exemplars Used by the Subcommittee
Subcommittee Members
Office of Standards, Assessments and Research & Evaluation
Purpose and Concepts• Reach consensus on the purpose of each
assessment. * Purpose statement includes, but may not be limited to,
definition/description of the type of assessment, value of assessment statement, what it measures, impact of this assessment on teaching and learning, SB 08-212 requirement, etc.
• Compile an initial draft of concepts for each assessment and prioritize.* May include, but may not be limited to, format, mode
of delivery, frequency of the assessment, duration of the assessment, delivery of results, etc.
*These are first drafts and can be revised through meeting 4.
Office of Standards, Assessments and Research & Evaluation
Formative AssessmentsPurposes from Meeting 1• To gauge student learning. Can occur anytime (embedded in
classroom practice)• As a check for what students are ready for next. Answers the
question, “are they ready to move on?”• Provides information for the next step for differentiation in
instruction • Results should be useable for students for self assessment and
peer assessment. Facilitates personal goal-setting and student motivation.
• Is diagnostic – should inform both teachers and students of how changes have been made over time. Builds a body of evidence that shows growth.
How has today’s conversations, exemplars and research affected these ideas?
Office of Standards, Assessments and Research & Evaluation
Formative AssessmentsAssessment Guidelines and Concepts from Meeting 1• PD to use formative assessments to measure standards• Evaluative vs. descriptive feedback – PD to use results to change
instruction, not necessarily for a “grade” – subcommittee to define tools and processes.
• PD to change expectations around how and why formative grades are used
• Developed at teacher/district level, PD and resources from the State level
• Exemplars of quality formative assessments. Can be done at the local level.
• Idea: Bank of common formative tasks.
How has today’s conversations, exemplars and research affected these ideas?
Office of Standards, Assessments and Research & Evaluation
Formative Definition• Process, not product• Aligned to standards• Informs instruction - diagnostic• Changes behavior of teachers and students• Provides student motivation towards goals• Not punitive• Small scale• Doesn’t compare – teacher to teacher, school to school,
district to district• "formative assessment is a process used by teachers and
students during instruction that provides feedback to adjust ongoing teaching and learning to improve students' achievement of intended instructional outcomes" (CCSSO, 2008, p. 3).
Office of Standards, Assessments and Research & Evaluation
Interim Assessments
Purposes from Meeting 1• To compare student performance between students in classrooms,
schools, and districts.• Informs system, programmatic changes vs. classroom instruction• Legislative purposes at the district level require the measurement
of growth and are used for improvement planning• Informs instructional augmentation.• To evaluate teachers, schools, districts
How has today’s conversations, exemplars and research affected these ideas?
Office of Standards, Assessments and Research & Evaluation
Interim Assessments
Assessment Guidelines and Concepts• Interim assessments need to be aligned with the State summative
assessment.• PD will be helpful for teachers to understand the use of data.• A through-course model , aligned to the state summative could
take the pressure off preparing for the summative. Concern: Over-testing
• Scoring needs to be standardized in order to be valid for any reporting purposes.
How has today’s conversations, exemplars and research affected these ideas?
Office of Standards, Assessments and Research & Evaluation
Interim Definition• (1) evaluate students’ knowledge and skills relative to a
specific set of academic goals, typically within a limited time frame, and (2) are designed to inform decisions at both the classroom and beyond the classroom level, such as the school or district level. (Achieve, Aspen Center)
• Growth towards mastery of standards over time• Aligned to standards• Results can be aggregated• Informs systems and programs• Allows teachers to communicate re: results and practice• Goal setting for students• Accessible to parents, families, and community educators
Office of Standards, Assessments and Research & Evaluation
Consensus Building – Fist to FiveFist A no vote - a way to block consensus.
I need to talk more on the proposal and require changes for it to pass.
1 Finger I still need to discuss certain issues and suggest changes that should be made.
2 Fingers I am more comfortable with the proposal but would like to discuss some minor issues.
3 Fingers I’m not in total agreement but feel comfortable to let this decision or a proposal pass without further discussion.
4 Fingers I think it’s a good idea/decision and will work for it. 5 Fingers It’s a great idea and I will be one of the leaders in
implementing it.
Office of Standards, Assessments and Research & Evaluation
Homework
1. What are the Benefits and Consequences of the Proposed Purpose and Concepts?
2. What are the roles and responsibilities of students, teachers, principals, districts, and the State in this type of system?
Come to the Grand Junction Meeting prepared to discuss these questions.
Office of Standards, Assessments and Research & Evaluation
Save the Dates:
Meeting 3 - August 28 – Grand JunctionMeeting: Two Rivers Convention Center, 159 Main St., Grand Junction, CO 81501Hotel: Doubletree Hotel Grand Junction, 743 Horizon
Dr., Grand Junction, CO 81506-3906. Please call: 970-241-8888 to make your hotel room
reservation.
Meeting 4 - September 11 – Fort CollinsThe Hilton Fort Collins, 425 West Prospect Road, Fort
Collins, Colorado 80526Phone: 970-482-2626
Office of Standards, Assessments and Research & Evaluation
Thank You!