the data disconnect between classroom tasks & …...the data disconnect between classroom tasks...
TRANSCRIPT
Facilitated by WISExplore Team Members:
The Data Disconnect Between Classroom Tasks & Standards
Leading for Learning Summit Breakout SessionJune 28, 2019
Jim Lee, CESA 12Lisa Arneson, CESA 3
1
Session Outcomes
● Understand how scores from misaligned tests can lead educators to make faulty inferences and predictions.
● Utilize new research that suggests educators should be concerned with alignment of instruction and assessment practices.
● Explore various data practices and tools that inform whether expectations, instruction, assignments, and learning align with grade-level standards.
● Plan an alignment investigation in their school.
2
Importance of Validity
Validity: The extent to which an assessment accurately measures what it is intended to measure.Content Validity: The tested content is aligned with what was taught. Content validity is increased when assessments require students to make use of as much of their classroom learning as possible.Predictive Validity: The extent to which a test score predicts performance on some criterion measure.
What are the consequences if test results are NOT VALID based on what we’ve taught?
4
Making Inferences
Valid measures give us the ability to make inferences from the data about what the student knows, can do or understands.“Based on these test scores, we can infer that the student
has mastered this content.”
“Validity is the degree to which all the accumulated evidence supports the intended interpretation of test scores for the proposed purpose” (AERA, 2008, p.11).
What happens when our inferences from test scores are incorrect?
5
Ensuring Validity
Test Content & Rigor
Adopted Curriculum &
Materials Content & Rigor
Standards Content & Rigor
Taught Curriculum
Content & Rigor
6
Example: Grade Level Misalignment
● All special ed students in 5th grade earned “Below Basic” on the Forward Exam
● We infer that they haven’t mastered tested content because the test is too hard for them. These students continue to be placed in remedial courses throughout their school career
● In reality, we find that students are pulled out of core classes and they are taught from reading and math materials that are at the 3rd grade level
● In this instance, the students are not exposed to grade-level instruction that is tested on the state test
Don’t go down a rabbit hole!
What can be done in this situation for
better alignment and student learning at
expected levels?7
See the Difference in ELA
CCSS.ELA-LITERACY.RI.3.8Describe the logical connection between particular sentences and paragraphs in a text (e.g., comparison, cause/effect, first/second/third in a sequence).
CCSS.ELA-LITERACY.RI.5.8Explain how an author uses reasons and evidence to support particular points in a text, identifying which reasons and evidence support which point(s).
What is a 5th grader taught that a third grader isn’t?Do these skills rely on a student’s ability to
read at the 5th grade level?
8
Faulty Inference from aTest ScoreTest Score:Dibels ORF (Oral Reading Fluency) number of correct words per minute: 16
https://dpi.wi.gov/sites/default/files/imce/standards/pdf/ela-stds-app-a-revision.pdf
Faulty Inference: Infer that the student has not mastered reading comprehension (informational text).
9
Grade 2 ELA Standards RI
Do No Harm
When your students sit down to take an assessment, will the task types be familiar
enough to boost their confidence?
If the test content and rigor are unfamiliar to the student -- what harm might it cause?
10
Research:Students Not Taught at Grade-level
“Low-income students, English learners, students of color and those with disabilities are far less likely to be given the chance to do grade-level work.” (Educationdive.com article)
12
Opportunity Myth
In the nearly 1,000 lessons observed:
● Students were working on activities related to class 88 percent of the time.
● They met the demands of their assignments 71 percent of the time, and ● More than half brought home As and Bs. ● Yet students only demonstrated mastery of grade-level standards 17
percent of the time. ● That gap exists because so few of their assignments actually gave them
the chance to demonstrate grade-level mastery.
Also, regarding teacher expectations:
● 82% of teachers support their state's standards, only 44% say they expect that their students can reach those standards.
(TNTP: Opportunity Myth) 13
“Myth” Found Four Things Missing in Schools
● Grade-appropriate assignments● Strong instruction● Deep engagement, and● Teachers who hold high expectations.
14
Fordham Institute
While many students get good grades in the subject, far fewer
earn top grades on end-of-course algebra tests, and grade
inflation has grown worse over time at schools serving more
affluent students.
(Fordham Institute: Grade Inflation in High Schools: 2005-2016)
15
Process for Alignment ValidationWhat information do you have?● Grade level standards● Content Curriculum● State Assessment (content) blueprint & technical guides● Classroom assessments, item specifications● Materials and/or lesson plans with activitiesGuiding Questions:1. Does the curriculum include the same content as what is tested in
the assessment?2. Is there a difference between the adopted curriculum and what is
taught in the classroom?3. Does the curriculum align with the standards? How do you know?4. Are the learning expectations for students at the same level of rigor
as the standards? 19
Tools to Examine Test Specifications
Provides:● Test blueprint● Test Maps● Item types and descriptions● Number of items/points● Standards alignment● Depth of Knowledge (DOK)
20
● Forward Exam Tech Specs● Aspire Summative Technical Manual● ACT Alignment Report● ACT Technical Manual Supplement● ACCESS for ELLs Interpretive Guide
Use technical manuals for:
● Deep content and DOK alignment
Allow ample time for thorough study.
Test Blueprints
● English Language Arts● Mathematics● Science● Social Studies
21
Use blueprints for:
● Distribution of content
● DOK levels
● Number of points
Allow ample time for study.
Data Inquiry Using Test Specifications
Assessment Technical Guides can help you answer:● How can we understand how the assessments were
constructed?● What types of tasks are students asked to complete or
answer?● What depth of thinking is required in the assessment?● How well does our curriculum align with the assessment
items?● Which standards are assessed more frequently? Where are
these taught and assessed in our curriculum? Are these identified as “power standards?”
22
Tools to Examine Instructional Materials
● EdReports● Achieve the Core’s
○ Materials Alignment Quick Check (ELA)
○ EQuIP Suite of Tools
○ Instructional Materials Evaluation Tool (IMET)
● CCSSO, Achieve, and Student Achievement Partners have jointly developed a Toolkit for Evaluating the Alignment of Instructional and Assessment Materials to the Common Core State Standards
● DPI IMPL resources
23
Data Inquiry Using Materials Alignment Tools
Instructional materials tools can help you answer:● To what degree is our materials aligned to grade-level
standards?● To what degree is our materials aligned to the district
adopted curriculum?● Are the materials designed to meet the diverse needs of all
students? (i.e. SwD, English Learners, Gifted, etc.)
24
Tools to Examine Curriculum Alignment● Wisconsin State Standards● Learning Progressions
○ Mathematics■ Arizona: Progressions Documents for the Common Core Math Standards ■ NC State: Learning Trajectories for the K-8 Common Core Math
Standards○ ELA
■ Delaware: ELA Learning Progressions ○ Science
■ Sense Publishers: Learning Progressions in Science○ Social Studies
■ Lewis Central College: K-12 Learning Progressions in Social Studies● Performance Level Descriptors
○ ACT Performance Level Descriptors (ELA, Math, Science)
○ Forward Performance Level Descriptors (ELA, Math) Science, Social Studies
○ DLM Performance Level Descriptions
25
Data Inquiry Using Curriculum Alignment Tools
Curriculum alignment tools can help you answer:● How well does the adopted curriculum align with the district
adopted standards?● To what degree is our curriculum aligned with the content
learning progressions?● What evidence is there that we teaching and assessing all
students with grade-level standards? ● Are our interventions and programs based on appropriate
standards learning progressions?● What is the rigor of what is taught and assessed?● Are there gaps in instruction and assessment?
26
Depth of Knowledge
From ELA 8th grade Item Sampler, P. 3
27
Sample Activity: Assessment Blueprint
28Item Number
Response Type
Assessment blueprints are a valuable alignment tool. They can be used to plan an assessment or to adjust an assessment.
Layers of Alignment
30
Layer 1 - Simple Alignment
Layer 2 - Alignment with Standards Rigor
Layer 3 - Alignment with Local Curricula and Resources
Layer 4 - Alignment with Classroom Instruction and Assessment
Sample Activity
Compare Forward Test Specs to Local Curriculum, Instruction, and Assessment at one grade level1. Open Forward ELA Test Specs and choose a grade to study2. Examine the spreadsheet. What do you notice?3. Which standards are tested multiple times on the assessment?4. Are these same standards assessed with the same frequency in the
grades above and below?5. Where are those standards taught and assessed in your curriculum? What
is your evidence of student mastery?6. Examine the Depth of Knowledge column. 7. How many items are assessed at a level 3 DOK?8. Which standards are assessed at a higher DOK level? Which ones aren’t?9. Where do local assessments include items at the same level of rigor?
32
Next Steps
Contact your local CESA for support with these activities.
Each has a data specialist to help you!
34
There are many ways to continue your learning after the Summit, that also
connect to this session!
35
37
● Multi-day Academies● Reimbursement for school
and district leaders through Title II
● Individual and team-based opportunities
● Direct connection to 3 tenets
Wrap Up & Feedback Form
38
Please take a few minutes to complete this feedback form to help us get better.