next generation assessments
DESCRIPTION
NEXT GENERATION ASSESSMENTS. K-12 Reinvention Symposium October 22-24, 2010. A Call for Next Generation Assessments. - PowerPoint PPT PresentationTRANSCRIPT
NEXT GENERATION ASSESSMENTS
K-12 Reinvention SymposiumOctober 22-24, 2010
A CALL FOR NEXT GENERATION ASSESSMENTS
“I am calling on our nation’s Governors and state education chiefs to develop standards and assessments that don’t simply measure whether students can fill in a bubble on a test, but whether they possess 21st century skills like problem-solving and critical thinking, entrepreneurship and creativity.”
-- President Barack Obama March 10, 2009
HOW THE DEMAND FOR SKILLS HAS CHANGEDECONOMY-WIDE MEASURES OF ROUTINE AND NON-ROUTINE TASK INPUT (U.S.)
1960 1970 1980 1990 200240
45
50
55
60
65
Routine manualNonroutine manualRoutine cognitiveNonroutine analyticNonroutine interactive
(Levy and Murnane)Mea
n ta
sk in
put a
s per
cent
iles o
f th
e 19
60 ta
sk d
istrib
utio
n
The dilemma of schools:The skills that are easiest to teach and test are also the ones that are easiest to digitize, automate, and outsource
CONCERNS ABOUT THE EFFECTS OF HIGH-STAKES MULTIPLE-CHOICE TESTS ON INSTRUCTION “I have seen more students who can pass the [state test] but cannot apply those skills to anything if it’s not in the test format. I have students who can do the test but can’t look up words in a dictionary and understand the different meanings…. As for higher quality teaching, I’m not sure I would call it that. Because of the pressure for passing scores, more and more time is spent practicing the test and putting everything in [the test] format.”-- A Texas Teacher
WORLDWIDE REFORM INITIATIVES Expectations for higher- order skills Rich content Modes of inquiry “Teach less, learn more” Project work, tasks requiring research,
analysis, application, self assessment, production
Performance tasks Assessment of, as, and for learning Learning progressions
COMMON PRACTICES ACROSS COUNTRIES Tightly integrated system Assessments include evidence of actual
student work Teachers integrally involved Assessments inform course grades, provide
information to colleges and employer, not to determine diploma
Assessments designed to continuously improve teaching and learning
HOW ASSESSMENT SYSTEMS AIM TO IMPROVE TEACHING AND LEARNING Together on-demand and curriculum
embedded assessments evaluate analytic & performance abilities – measure the full range of knowledge and skills
Moderated teacher scoring Learning progressions (shape curriculum,
personalizes learning, know where the student is
School based assessments model good instruction
THE CHALLENGEHow do we get from here... ...to here?
All students leave high
school college and
career ready
Common Core State Standards
specify K-12 expectations for
college and career readiness
AND HOW CAN ASSESSMENT CONTRIBUTE?
How do we get from here...
All students leave high
school college and
career ready
Common Core State Standards
specify K-12 expectations for
college and career readiness
...to here?
RACE TO THE TOP ASSESSMENT PROGRAM COMPETITION $350 million of Race to the Top Fund set aside for
awards to consortia of states to design and develop common K-12 assessment systems aligned to common, college- and career-ready standards
Competition asked consortia to design assessment systems that meet dual needs of: Instructional improvements Accountability
In September 2010, the U.S. Department of Education awarded two grants: Partnership for Assessment of Readiness for College and Careers
(PARCC) Smarter Balanced Assessment Consortium (SBAC)
The winning consortia have four years to develop assessment systems, and participating states will administer new assessment statewide by 2014-2015
Partnership for Assessment of College and Career Readiness (PARCC)
12
PARCC STATES
PARCC STATES11 Governing States –Arizona–District of Columbia–Florida-Fiscal Agent–Illinois–Indiana–Louisiana–Maryland–Massachusetts-Board Chair–New York–Rhode Island–Tennessee
13
15 Participating StatesAlabamaArkansasCaliforniaColoradoDelawareGeorgiaKentuckyMississippi
New HampshireNew JerseyNorth DakotaOhioOklahomaPennsylvaniaSouth Carolina
BACKGROUNDSTATES INVOLVED
Fiscal Agent: Washington State
Governing AdvisoryCT, HI, ID, KS, ME, MI, MO, MT, NC, NM, NV, OR, UT, VT, WA, WI, WV
AL, CO, DE, GA, IA, KY, ND, NH, NJ, OH, OK, PA, SC, SD
17 14Total Number of States = 31
GOALS OF CONSORTIUMPARCC
States in the Partnership are committed to building their collective capacity to increase the rates at which students graduate from high school prepared for success in college and the workplace.
SMARTER
To ensure that all students leave high school prepared for postsecondary success in college or a career through increased student learning and improved teaching.
18
PARCC ASSESSMENT SYSTEM DESIGN: MEASURING MASTERY OF THE COMMON CORE STANDARDS
ELA/LiteracyReading complex literary and informational textWriting – both on demand and over timeResearchSpeaking and listeningLanguage , conventions, vocabularyMathematicsConceptual understanding and procedural fluencyApplication via modeling and strategic problem solvingReasoning, explaining, justifying
19
PARCC ASSESSMENT SYSTEM DESIGN: MEASURING MASTERY OF THE COMMON CORE STANDARDS
ELA/LiteracyReading complex literary and informational textWriting – both on demand and over timeResearchSpeaking and listeningLanguage , conventions, vocabularyMathematicsConceptual understanding and procedural fluencyApplication via modeling and strategic problem solvingReasoning, explaining, justifying
20
PARCC ASSESSMENT SYSTEM DESIGNThe through-course components in both subjects
will be administered after approximately 25 percent, 50 percent and 75 percent of instruction
ELA 1 and ELA 2 One or two tasks involving reading, drawing conclusions and presenting analysis in writing
Math 1 and Math 2 One to three tasks that assess one of two essential topics in mathematics (standards or clusters of standards)
PARCC ASSESSMENT SYSTEM DESIGN
ELA 3: Performance tasks) that require evaluating information from within a set of digital resources, evaluating their quality, selecting resources, and composing an essay or research paper
ELA 4 (speaking and listening) Students will present their work from ELA 3 to classmates and respond to questions. Teachers will score, using a standardized rubric, and can use results in determining students class grades.
Math 3 Performance tasks) that require conceptual understanding, procedural fluency, and application of mathematical tools and reasoning.
PARCC ASSESSMENT SYSTEM DESIGNEnd-of-Year – Comprehensive, computer-scored
assessment that includes a range of item types, including innovative, technology-enhanced items. Enables quick turnaround of student work.
A students summative score – used for accountability purposes – will include his/her performance on Through-Courses 1, 2, and 3 as well as the end of year assessment
23
PARCC ASSESSMENT SYSTEM DESIGN:FORMATIVE ASSESSMENTFormative Tools “[Thoughtful, curriculum-aligned, and valid ways of
determining what students know, rather than leaving the burden of planning and assessing on the teacher alone.” Shavelson et al. (2008):
Partnership Resource Center (PRC)―an online, digital resource that includes two supports:
released items with item data, student work, rubrics model curriculum frameworks
Text Complexity Diagnostic Tool: a computer adaptive tool to identify students’ proximate zone of development and supply suggestions for appropriate texts for students to read
K-2 Assessments in ELA/Literacy and Mathematics
24
PARCC ASSESSMENT SYSTEM DESIGNData to Support Instruction, Professional
Development, and Accountability Decisions– Types of Data
• Student achievement scores and growth measures• Information on readiness for college and careers • Item analysis of released items
– Reporting• Periodic Feedback Reports• Annual Stakeholder Reports• Item Analysis Reports• Interactive Data Tool (online)
– Available data will be aggregated at a level appropriate for each audience and report
25
PARCC ASSESSMENT SYSTEM DESIGN
Accessibility and AccommodationsStudents with disabilities and English Learners will be
considered from onset of development process Items and test forms will be created using an evidence
centered design approachUniversal design methods will be considered in every step of
the processAccessibility and Accommodations committee will be formed
to advise Partnership
PARCC TIMELINE
October 2010 Launch and
design phase
September 2011
Development and Design
begins
September 2012
First year field testing and
related research and
data collection
begin
September 2013 Second
year field testing begins
and related research and
data collection continues
September 2014 Full
administration of PARCC assessment
begins
Summer 2015 Set achievem
ent levels,
including college-ready
performance
levels
http://www.fldoe.org/parcc/
Partnership for Assessment of Readiness for College and Careers
HISTORICAL DEVELOPMENT OF THE SMARTER BALANCED CONSORTIUM
SMARTER
MOSAIC
Balanced
• Computer Adaptive
• Formative Capacity
• Integrated System
THE PURPOSE OF THE CONSORTIUM
To develop a set of comprehensive and innovative assessments for grades 3-8 and high school in English language arts and mathematics aligned to the Common Core State Standards.
The assessments shall be operational across Consortium states in the 2014-15 school year.
THEORY OF ACTION• A model of verifiable accomplishments/milestones,
leading to the desired outcome
• Accomplishments/milestones are inter-dependent
• The theory of action is closely linked to the validation argument for the assessment system
31
The SMARTER Balanced Theory of Action
All students leave high
school college and
career ready
Summative adaptive
assessments are benchmarked to college & career
readiness
Technology supports
innovative & comprehensi
ve assessments
Technology provides increased access to learning
State policies and practices
support increased
expectations
Common Core State
Standards specify K-12 expectations
for college and career
readiness
Clear communication of
expectations to stakeholders
Professional
capacity-building
PD and other supports for teachers to instruct on the CCSS
Teachers design and
score assessment
items & tasks
Teachers use formative tools and
practices to improve
instruction
Interim/Benchmark assessments are used as progress
checks
32
The SMARTER Balanced Theory of Action
All students leave high
school college and
career ready
Summative adaptive
assessments are benchmarked to college & career
readiness
Technology supports
innovative & comprehensi
ve assessments
Technology provides increased access to learning
State policies and practices
support increased
expectations
Common Core State
Standards specify K-12 expectations
for college and career
readiness
Clear communication of
expectations to stakeholders
Professional
capacity-building
PD and other supports for teachers to instruct on the CCSS
Teachers design and
score assessment
items & tasks
Interim/Benchmark assessments are used as progress
checks
Teachers use formative tools and
practices to improve
instruction
THEORY OF ACTIONAssessment system that balances
summative, interim/benchmark, performance, and formative components for ELA and mathematics:
1.Computer adaptive assessment system/summative
a. Grades 3–8 and 11 (testing window within the last 12 weeks of the instructional year)
b. Selected response, enhanced constructed response, technology enhanced, and performance events (grades 3–8, 2/year: HS up to 6)
2.Adaptive interim/benchmarka. Learning progressionsb. Administered throughout the year
3.Formative Processes and Tools
ASSESSMENT DESIGNThe Consortium will provide the following by
the 2014-15 school year:3. Formative tools and resources4. Responsible flexibility5. Distributed summative assessment
a. Content clusters throughout a courseb. Most appropriate time for each studentc. Scores rolled up
Smarter Balanced Assessment Consortium
35 AFT October 20, 2010
Assessment System Highlights
SYSTEM HIGHLIGHTSSummative assessments using online computer
adaptive technologies• Efficiently provide accurate measurement of all students,
across the spectrum of knowledge and skills• Incorporate adaptive precision into performance tasks
and events• Will assess full range of CCSS in English language arts and
mathematics• Describe both current achievement and growth across
time, showing progress toward college- and career-readiness• Scores can be reliably used for state-to-state comparability,
with standards set against research-based benchmarks • The option of giving the summative tests twice a year.
SYSTEM HIGHLIGHTSOptional interim/benchmark assessments
• Are aligned to and reported on the same scale as the summative assessments
• Help identify specific needs of each student, so teachers can provide appropriate, targeted instructional assistance
• Incorporate significant involvement of teachers in item and task design and scoring
• Are non-secure and fully accessible for use in instruction and professional development activities
• Provide students and teachers with clear examples of the expected performance on common standards.
SYSTEM HIGHLIGHTSDigital library of formative tools
Instructionally sensitive, on-demand measures that enable differentiation of instruction
Use is associated with improved teaching and increased student learning
SYSTEM HIGHLIGHTSComputer Adaptive TechnologyAllows for the breadth of the
Common Core State Standards to be comprehensively assessed while minimizing test length.
Allows increased measurement precision relative to fixed form assessments.
Allows items to be presented as a function of student ability as measured during the test.
SYSTEM HIGHLIGHTSOnline, tailored reporting system
• Supports educator access to information about student progress toward college- and career-readiness
• Allows for exchange of student performance history across districts and states
• Uses a Consortium-supported backbone, while individual states retain jurisdiction over access permissions and front-end “look” of online reports.
SYSTEM HIGHLIGHTSBenefits and efficiencies from
“economies of scale” due to a multi-state consortium • Cost savings• Shared interoperable open source software
platforms: Item generation, item banking, and adaptive testing no longer exclusive property of vendors
• Common, agreed-upon protocols for accommodations for students with disabilities and ELL students.
43 AFT October 20, 2010
Professional Capacity and
Outreach
45 AFT October 20, 2010
Collaboration with Higher Education
IHE COLLABORATIONGOAL: Better prepare students for college-
and career-readiness.
Collaborate with IHEs to create student achievement standards that define college-ready.
Students will enter IHE systems having met common, clear college-ready standards.
Students will be able to track readiness for college and careers throughout high school.
TIMELINE
TO FIND OUT MORE......the SMARTER Balanced Assessment
Consortium can be found online at
www.k12.wa.us/SMARTER
CCSS APPENDIX BText Exemplars
Stories/LiteraturePoetryDramaInformational (ELA, Science, Social Studies)
Performance Assessment Examples
PROFESSIONAL CAPACITY BUILDINGCollaboration with existing professional
development networks in each participating State in order to:• Create a sustainable structure that supports teaching,
learning, reliable scoring of performance assessments, and development of formative tools and processes, and
• Foster interdependence and opportunities for shared learning among teachers and school leaders across States.
PROFESSIONAL CAPACITY BUILDINGCollaboration with existing professional
development networks in each participating State in order to:• Provide access to authentic learning experiences
among teachers and school leaders, and• Bring added efficiency and cost effectiveness to
professional capacity building activities.
DEVELOPING A TRANSITION STRATEGY Rigor/Relevance framework
Natural fit for Common Core Standards Emphasis on Deep knowledge and applied real
world situations Designing curriculum/instruction/assessments
RIGOR/RELEVANCE IN ASSESSMENTQuadrant A – Acquisition (Low Rigor/Low
Relevance
Use verbs synonymous with recall and understanding Call for explanation of knowledge or skill
Multiple choice, true/false or short answerRequire single standard response
RIGOR/RELEVANCE IN ASSESSMENTQuadrant B – Application (Low Rigor/High
Relevance)Verbs synonymous with recall, understanding, or applicationApplication of knowledge to real-world situationsPerformance-basedFollow a routine or set of procedures
RIGOR/RELEVANCE IN ASSESSMENTQuadrant C – Assimilation (High
Rigor/Low Relevance)Verbs synonymous with analysis, synthesis or evaluationCall for the explanation of knowledge or skill, but not real-world applicationInclude multiple step problemsEssays, presentations, or portfolios
RIGOR/RELEVANCE IN ASSESSMENTQuadrant D – Adaptation (High Rigor/High
RelevanceUse verbs synonymous with analysis, synthesis or evaluationCall for unique solutions in applying knowledge to real world problemsPerformance-basedInclude multi-step problems
GRADE 4/NAEP Sam can purchase his lunch at school. Each
day he wants to have juice that costs 50¢, a sandwich that costs 90¢, and fruit that costs 35¢. His mother has only $1.00 bills. What is the least number of $1.00 bills that his mother should give him so he will have enough money to buy lunch for 5 days?
GRADE 4/NAEP
RELEASED GRADE 4/WRITING Situation: The local newspaper is having a “Good
Friend” contest. To enter your friend, you must think of an event in your life when your friend did something with you or for you that showed what a terrific friend he or she is.
GRADE 4 Writing Task: Select your friend. (Remember, a friend
could be a child your age or a grownup.) Choose an event that shows how your friend is a good friend to you. Write a letter to the newspaper that tells about that event so that people will know why your friend deserves to win.
GRADE 7Situation: Not only do games have rules, but there are
also rules of respect and safety at home and at school. These rules help everyone understand what to do, how and when to do the activities, and how to be safe. Think of a time that you needed to know the rules for an activity. What happened? Why were the rules important? Could someone else learn from your experience? Your school newspaper is running a series of first-person articles about lessons students have learned.
GRADE 7 Writing Task: In an article for your school newspaper, tell
about a time when knowing the rules was important.
STRATEGY TO CONSIDER Step 1: By content area, identify assessment
anchors that are a natural fit for each content area. Be hard-nosed. Only select what you are willing to teach until students learn it!
Step 2: School wide, track the “adoption” of assessment anchors, identify orphans, and make adjustments as needed.
Step 3: By course, design curriculum, instruction, and assessment to teach students to be proficient in all assessment anchors assigned to specific content areas/courses.
RELEASED ITEMSResources
State released itemsNAEPDepth of Knowledge Levels 3 & 4Assessment Kits
DEVELOP A THOUGHTFUL PLAN OF ACTION