Download - Learning Progressions in Science
Learning Progressions Learning Progressions in Science
Context
Conference in Iowa City, IA in June 2009Edited book based upon conference papers & conference papers & synthesis discussionsFocused on challenges in 4 g 4strands of learning progressions work :
DefiningDefiningAssessingModeling
National Science FoundationAward No. DRL-0824432
Using
Potential Role of Learning Progressions (LPs)
Grounding assessments in research on student glearning
Aligning curriculum, instruction, and assessment around “big ideas”
Focusing on a smaller number of “big ideas” with ti it d l lcontinuity across grade levels
Describing student learning in terms of qualitatively changing knowledge and practice rather than changing knowledge and practice rather than mastery of a checklist of standards or benchmarks
Consideration of Assessment at a Range of Levels
Classroom Gotwals, Songer, & Bullard: Classroom-Level Research
District
StateState
Alonzo, Neidorf, & Anderson: Large-Scale Assessment
National
International
Using Learning Progressions to Using Learning Progressions to Inform Classroom Assessmento C ass oo ssess e t
A melia Wenk Gotwals , Michigan State Universi ty, g y
Nancy Butler Songer & Lea Bul lard, Universi ty of Michigan
Overview
Potential Role of Learning Progressions in g gClassroom Assessment
ChallengesDesigning Items for Formative Use
Coding Non-conforming Student Responses
C l i d R d tiConclusions and Recommendations
Role of Learning Progressions for Classroom AssessmentAssessment
Provide coherence between curriculum, assessment ,and professional development
Provide a “vision” for teachers to see how students may develop more sophisticated ideas
Design assessment opportunities to gather evidence f h t d t f ll l i iof where students fall on a learning progression
Use evidence to inform future instruction
Fundamental Challenges/Questions
How can we design classroom assessment items such gthat students at a range of placements on a learning progression have opportunities to provide evidence
f h t th k d d ?of what they know and can do?
Wh t d ith t d t th t d t What can we do with student responses that do not “fit” into our hypothesized learning progression?
What can these responses tell us about:What can these responses tell us about:The students?
The learning progression?
h iThe assessment items?
Deep Think Learning Progression
Three grade levels4th grade• 4th grade
• 5th grade• 6th grade
Th i t t d Three interconnected content strands• Ecology (E)• Classification ( C )• Biodiversity (B)
Focus on scientific Focus on scientific explanations• Claim
Evidence• Evidence• Reasoning(Songer, Kelcey, & Gotwals, 2009)
Scientific Explanations
Claim: Assertion or l i i conclusion answering
scientific question
E id S i tifi d t Evidence: Scientific data to support the claim
Reasoning: Justification Reasoning: Justification linking claim and evidence using scientific evidence using scientific principles
( l i 8 h (Toulmin 1958, Kuhn 1993, Lee 2003, Songer, Kelcey & Gotwals, 2008)
Explanation difficultiesp
Claims: difficulty articulating clear claims based Claims: difficulty articulating clear claims based on data; don’t back up claims with evidence or reasoning
Evidence: what counts as evidence (appropriate); sufficient evidence; convincing evidence
Reasoning: doesn’t explicitly link evidence to claim; not articulated
Simplified Scientific Explanation Learning Progression Simplified Scientific Explanation Learning Progression ((SongerSonger, , KelceyKelcey & Gotwals, 2009; Gotwals & & Gotwals, 2009; Gotwals & SongerSonger, 2010), 2010)
Level 4: Student constructs scientific explanationp
Level 3: Student makes a claim and backs it up with sufficient and appropriate evidence but does not use reasoning to tie the evidence to the claimLevel 2: Student makes a claim and backs it up with evidence but the evidence is insufficient orevidence, but the evidence is insufficient or inappropriateLevel 1: Student makes a claim but does not back it up pwith evidence
Level 0: Student does not make a claim or makes an incorrect claim
Assessment DesignAssessment Design
What are features of ideal LP assessments? What are features of ideal LP assessments?
gg
What are features of ideal LP assessments? What are features of ideal LP assessments?
More than end point tasks; e.g., assessments that are sensitive to progress towards end goalp g g
Assessments that provide a range of information of middle knowledge (vs. all or nothing) g ( g)
Assessments that provide information for a range of ability and age levels
Assessments that articulate the when of middle knowledge development (e.g., continuous)
Types of ItemsTypes of Items
4 levels of items4 levels of items
1 Minimal Items: Multiple choice1. Minimal Items: Multiple choice
2. Intermediate Level Items: Content and Explanation ScaffoldsExplanation Scaffolds
3. Intermediate Level Items: Explanation OR Content Scaffolds onlyy
4. Complex Items: No Scaffolds
Type 2: Content & Explanation ScaffoldsType 2: Content & Explanation ScaffoldsTYPE OF ORGANISM WHAT THEY USE FOR ENERGY
small fish water lily large fish small fish, water lily h ll fi h l fi h iheron small fish, large fish, insects
Write a scientific explanation for the following questionScientific Question: Is the large fish a producer or consumer?
Hint:Think about how producers and consumers get
energy.
Make a CLAIM:Write a sentence that answers the scientific question
Give your REASONING:Write the scientific concept or definition that you thought about to make your claim
Hint:Think about the definition of the scientific term
Give your EVIDENCE:Look at your data and find two pieces of evidence that help answer the scientific question
you used.
Hint:Think about
where the largethe scientific question.1. 2.
where the large fish get its energy.
What Information LP Assessment System ProvideWhat Information LP Assessment System Provideyy
Assessment items & coding rubric based on our Assessment items & coding rubric based on our learning progression allowed us to gather rich evidence of students’ abilities to formulate scientific explanations with and without scaffolding
Using the same scaffolding and coding rubric over multiple years allowed us to trace students’
i progress over time
Student responses that do not “fit”Student responses that do not “fit”Student responses that do not fitStudent responses that do not fit
Student responses that do not “fit”Student responses that do not “fit”pp
ClaimClaim
Student responses that do not “fit”Student responses that do not “fit”Student responses that do not fitStudent responses that do not fit
ReasoningReasoning
Conclusions and Recommendations
ConclusionsImportance of assessment tasks that are sensitive to progress towards an end goal and nuances in middle knowledge
Additional researchMaking sense of responses that do not fit neatly into the LPMaking sense of responses that do not fit neatly into the LP
Continued research on scientific explanation formulation across content areas and grade bands
h d id h f i fTeachers use LPs to decide what formative assessment format to give to individual students
Using Learning Progressions to Using Learning Progressions to Inform Large-Scale Assessmento a ge Sca e ssess e t
Alicia C. Alonzo, Michigan State Universi ty
Teresa Neidorf , American Inst i tutes for Research
Charles W. Anderson, Michigan State Universi ty
CCSSO National Conference on Student Assessment, June 21, 2010
Fundamental Challenge/Question
How might learning progression (LP)-based How might learning progression (LP) based items fit into a large-scale testing apparatus which has been designed to make different which has been designed to make different claims about student achievement?
Challenges Recommendations
C t ti F hContrasting purposes: Policy decisions
Contrasting practices:
For researchers
For test-developersg pTechnical decisions
p
Contrasting Purposes: Policy Decisions
Current large-scale assessment systemsg yClaims about students’ mastery of particular benchmarks for their grade level (often expressed in terms of achievement levels)levels)
LP approachClaims about where student performances fit in a succession of pincreasingly sophisticated ideas and practices (which may span multiple grade levels)
LP levels articulate ideas and practices characteristic of students LP levels articulate ideas and practices characteristic of students at a given level of sophistication (including non-canonical ways of thinking) with respect to a particular “big idea”
Contrasting Practices: Technical Decisions
Item Development
Pilot Testingg
Item Analysis and Evaluation
Design of Operational Assessmentsg p
Scoring and Reporting
Item Development
Current Large-Scale Assessment SystemsCurrent Large-Scale Assessment Systems
LP ApproachLP ApproachAssessment SystemsAssessment Systems
Linear process Iterative process
Framework (LP)Framework o ( )o
Individual items designed Sets of items designed to
Assessment Student Responses Assessment
to make judgments about students’ mastery of particular knowledge/skills
evaluate the sophistication of students’ thinking with respect to the topic of a
i l p g /
Traditional item types and scoring procedures
particular LPLP-specific item types
and/or scoring procedures
Possible LP-Based Item Types
Ordered Multiple Choice (OMC) (Briggs, Alonzo, Wilson, & Schwab, 2006)p ( )Appear similar to typical MC items, but options are mapped directly to levels of a LP
M lti l T F lMultiple True-FalseStudents are presented with a list of objects or properties and are asked to determine whether each of these meets a given criteriong
Constructed Response
Choose & ExplainpStudents are asked to choose from a set of options (such as in a multiple-choice format) and then to explain their choice
Pilot Testing
Pilot testing offers opportunities for data collection g ppfrom relatively large, representative sample of the target population
LP-Specific RequirementsMultiple pilot-testing phases
Pilot test design in which students respond to sets of LP based Pilot-test design in which students respond to sets of LP-based items
LP-based items cannot be randomly distributed through test b kl tbooklets
Larger pilot samples may be required
Item Analysis & Evaluation
Sets LP-based items must be analyzed to inform yevaluation of the items and the LP
Criteria for evaluating items for inclusion in typical large-scale tests are well established but may not be appropriate or adequate for evaluating LP-based itemsitems
Item difficulty
Item discrimination
Frequency distributions
Inter-rater reliability
iff i l i f i i ( )Differential item functioning (DIF)
Design of Operational Assessments
Issues and challenges in incorporating LP-based items into current assessment systems
Content coverage
Item presentation
Computer-adaptive testing (CAT) may be a promising vehicle for incorporating LP-based itemspromising vehicle for incorporating LP based items
Scoring & Reporting
LP-Specific ReportingLP-Specific ReportingIncorporating LP-Based Items into the Main ScaleIncorporating LP-Based Items into the Main ScaleItems into the Main ScaleItems into the Main Scale
Scaling (cut-score establishment) requires
LP-based items may include content not establishment) requires
cross-grade analyses
Different
include content not included in the assessment framework f i d l l
measurement/ reporting models may b i d f
for a given grade levelScore mapping or rescoring may be required
be required for LP-based frameworks
Reliable sco es ma be
Implications for reporting trends
Implications item release Reliable scores may be difficult to achieve
Implications item release policies
Recommendations
For researchers
For test developersp
Recommendations: Researchers
Research to develop well-grounded LPsCurriculum-neutralTargets grade levels typically assessed by large-scale testsIncludes students representative of target populations Includes students representative of target populations (rather than convenience samples)Longitudinal (rather than cross-sectional)
Additi l hAdditional researchTo explore/evaluate alternative measurement modelsTo inform design of LP-based assessmentsgTo explore impact of LP-based test designTo develop CAT administration of LP-based itemsTo inform reporting methodsTo inform reporting methods
Recommendations: Test Developers
Clear instructions and training for item-writers and gitem-scorers
Additional cycles of revision and pilot testing (with particular requirements)
Alternative methods for evaluating item quality
Alternative measurement models for score reporting
For More Information
LeaPS conference & book
http://education.msu.edu/projects/leaps/p // /p j / p /
Amelia W. Gotwals
Alicia C. Alonzo
alonzo@msu [email protected]