how’s it growing? enhancing data literacy and engagement through data visualization
DESCRIPTION
How’s it Growing? Enhancing Data Literacy and Engagement through Data Visualization. Juan D’Brot Executive Director Office of Assessment and Accountability. October 22 , 2012. The Overview. What is Growth and a Growth Model? Growth at the school level Applying growth (FAIPs) - PowerPoint PPT PresentationTRANSCRIPT
How’s it Growing?Enhancing Data Literacy
and Engagement through Data Visualization
Juan D’BrotExecutive Director
Office of Assessment and Accountability
October 22, 2012
The Overview1. What is Growth and a Growth Model?2. Growth at the school level 3. Applying growth (FAIPs)4. Drilling back down to the student level 5. Applying growth in the classroom6. Where to go next to drive instructional decisions
Why a Growth Model?
•AYP vs. Continuous Improvement (non- accountability) •Enhancing Data Literacy•Enhancing Engagement•Future-oriented Capacity Building– SMARTER Balanced Assessment Consortium– ESEA Reauthorization
Why NCLB Promotes Insensitivity to Individual Student Growth and School Improvement
No Content Mastery
Full Content Mastery
No Content Mastery
Full Content Mastery
Student Proficiency
AYP (status) Model
Growth (improvement) Model
Access to Full Spectrum of Student Performance and Improvement
Compliance vs. Continuous Improvement
Adequate Yearly Progress Improvement and Growth
•Snapshot•Non-comparable trends•Funding considerations•“Outcome”
•Multiple Snapshots•Comparable trends•(More) instructional consideration•“Process”
Transitioning to Growth
• Past: Contextualizing prior performance
• Present: Understanding current student and school growth
• Future: Building capacity for understanding and applying growth post-WESTEST 2
The WV Growth Model
•What are we trying to answer?•Purpose •How it works•What it yields?•How we interpret it•What levels (overview)– Past, present, future– Student, Grade, School, District, State– Prior growth, observed growth, projected growth (Growth
to Standard)
Asking the Right Questions…
• Until you’ve defined the question, you cannot examine the appropriate data.
• Why?– Different data answer different
questions – Different questions lead to different
conversations
Fact: …increased use of student assessment data as a “significant factor” in (insert favorite accountability topic here…)
• But what student data? That leads us to start asking questions…
• But what questions?
• Whose questions?
Whose Questions?• Researchers
• Administrators
• Teachers
• Parents
• Policymakers
• Business Leaders
• Community Members
• Guardians
WHY?To answer questions about the
student
How?By not declaring a verdict
(status). Instead, by changing the
conversation (to growth and progress)…
Fact: Assessment Systems try to be Everything to Everyone• Danger: answering too many
questions with the same data– How to Differentiate Instruction– Determining Student Proficiency– Measuring Class Performance– Determining AYP
• Possible Solution: Can we come up with a common question across stakeholders? But first…
The WV Growth Model: Our Purpose
• To provide an answer to a few common questions: 1. “How much did my student grow this
year?” 2. “What does this growth mean
compared to everyone else?”3. “Is it enough growth?”
What do these three questions have in common?
Information about students
Status vs. Growth
• Before growth, let’s review terms
– Status: A snapshot measure of a single point in time – “Proficiency”
– Growth: Multiple snapshots across many points in time
Status vs. Growth A Balanced Approach
Low Growth High GrowthQuadrant 2
High achieving but potentially declining: previously
unrecognized under status model
Quadrant 1High achievement and growth: A potential site of interest for best
practices
Quadrant 3Low achievement and growth:
Legitimately requires improvement
Quadrant 4Low achieving but improving:
previously unrecognized under the current status model. May be a potential site of interest for best
practices.
High Performing
(Status)
Low Performing
(Status)
Let’s Consider Growth at a High Level…
Measuring Growth: A Primer
• What is the likelihood of current (and future) performance given past performance?
• Based on students who have similar academic histories (academic cohort)
• Uses all available data to answer these three questions:– Am I proficient? (Status)– How much did I grow? (SGPs)– Is it enough growth? (Growth to Standard)
Our Growth Options (Primer, cont’d)• How much growth? – Starts with the
student. Can be aggregated to drive school conversations– High Growth (a good surprise)– Typical Growth (a statistical expectation)– Low Growth (a bad surprise)
• Is it enough growth? – Can drive student learning conversations– Catching Up– Keeping Up – Falling Behind
Questions - Revisited1. “How much did my student grow this year?”
(Time 2 – Time 1) – We can see this today
2. “What does this growth mean compared to everyone else?” (Normative Component)
3. “Is it enough growth?” (Criterion Component - CSOs)
For more information…Please visit the WVDE’s Growth Model webpage:
http://wvde.state.wv.us/growth/
Understanding Grade Level and School Growth Reports
Viewing Aggregated Growth at the School Level
• What to consider?– High vs. Low Achievement
• Individual Student Performance• Overall Percent Proficient
– High vs. Low Growth• Individual Student Growth• Overall Growth for a School
• An example of a grade and school level report
Grade Level Bubble Chart
• Growth Percentile vs. Scale Score (Achievement) for all students within a grade that have growth data
• Shows distribution of a group of students
• Hover capability to view student information at each data point (bubble)
Above Mastery
Mastery
Partial Mastery
Novice
Scale Score
Individual Student Growth Percentile
Each bubble is a single student who is plotted by:
Higher Observed Growth of a Single Student in a School
High
er P
erfo
rman
ce o
f a S
ingl
e St
uden
t
John Smith:2010-11 student Growth Percentile (Target): 95 (14)2010-11 Achievement Level (Scale Score): Distinguished (757)
Distinguished
Above Mastery
Mastery
Partial Mastery
Novice
Why are we interested in these data?
1. Easy to read overview of grade level performance
2. Growth + performance is more telling than performance alone
3. Ease of access can lead to immediate interest into student performance
4. Helps support our next logical question…
School Growth and Achievement Bubble Plots
School Growth and Achievement Bubble Plots
Enhancing Interest in the Story Behind the Data
Growth Model Interest in Data Examining Individual Performance Reviewing Individual Item Analysis Consideration of how to more effectively use of interim assessments and formative processes Validating what we see students learning and understanding
Teacher expertise leads to professional judgment about instruction
Examining District-LevelStudent Reports
• Log in to WVEIS and go to the Private Data Site
• At the bottom of the screen, you will see a link titled “Growth Model”
• Click Growth Model, select your district and select a school of interest.
• Click Go!
Growth Legend
Content Areas
Math (pdf) Reading (pdf)
Math (pdf) Reading (pdf)
Math (pdf) Reading (pdf) Click Here for Bubble Plots
Growth LegendSubgroup Comparison
Content Areas
Understanding Individual Student WESTEST 2 Growth
& Achievement Report
Achievement: A Quick Review• Current WESTEST 2 Student Report
shows: 1. Single Scale Score in each content area2. Level of Achievement (Performance
Levels)• Distinguished• Above Mastery• Mastery• Partial Mastery• Novice
The WV Growth Model: Our Purpose
• To provide answers to a few common questions: 1. “How much did the individual student
grow this year?” 2. “What does this growth mean
compared to everyone else?”3. “Is it enough growth?”
What do these three questions have in common?
Information about students
Measuring Growth: A Primer
• What is the likelihood of current (and future) performance given past performance?
• Based on students who have similar academic histories (academic cohort)
• Uses all available data to answer these three questions:– Am I proficient? (Status)– How much did I grow? (SGPs)– Is it enough growth? (Growth to Standard)
Our Growth Options (Primer, cont’d)• How much growth? – Starts with the
student. Can be aggregated to drive school conversations– High Growth (a good surprise)– Typical Growth (a statistical expectation)– Low Growth (a bad surprise)
• Is it enough growth? – Can drive student learning conversations– Catching Up– Keeping Up – Falling Behind
Hello. It’s nice to meet you.
Hello, Growth Report
Meet the Student Growthand Achievement Report
Novice Partial Mastery Partial Mastery Mastery
Name of Student Name of School
Interpretation Guide
Suggested Uses
Growth and Achievement for Math
Growth and Achievement for R/LA
Novice Partial Mastery Partial Mastery Mastery
Legend for Math
Legend for R/LA
Scale Scores for Math
Scale Scores for R/LA
Grade Level and School Year
Grade Level and School Year
Student Achievement Levels
Student Achievement Levels
WESTEST 2 Achievement Levels
WESTEST 2 Achievement Levels
Novice Partial Mastery Partial Mastery Mastery
GrowthLevelsGrowthPercentiles
The colored arrows between the dots represent growth from one grade to the next.
The color of the arrow signifies the level of growth.
High growth is indicated in green, typical growth in white, and low growth in red.
The fan represents how the student might score next year under several growth scenarios.
How much growth?
Is it enough growth?
How much growth?Growth Percentile – 1% to 99%• 66%-99% = High Growth = Green Arrow
- Good surprise - the student exceeded what was expected• 35%-65% = Typical Growth = White Arrow
- Statistical expectation - the student grew as
expected• 1%-34% = Low Growth = Red Arrow
- Bad surprise – The student grew less than was expected
Is it enough growth?Will a student be proficient next year?• Determined at only one grade level at a time• Depends on the achievement level where the student
was at a particular grade level• The fan can help us understand potential stories for a
given student• What level of growth is required for a student who
is proficient to keep up with proficiency next year? • What level of growth is required for a student who
is not proficient to catch up to proficiency next year?• What level of growth for any student, would cause
a student to fall or stay below proficiency next year?
Prior Proficient: Student is Keeping Up
Prior Proficient: Student is Not Keeping Up/Falling Behind (Math)
Prior Proficient: Student Keeping Up (Math)/Will Fall Behind (R/LA)
What about this student?
Novice
What about this student?
Turning to Companion Reports
–WVEIS-based Reports»Teacher-based Growth Report»CIAS/CSO Analysis»Teacher-based CSO Analysis»Student’s Individual Responses
A Sample WVEIS on the WEB Teacher Report
WESTEST 2 Growth Model Report
WESTEST 2 CSO Analysis
WESTEST 2/CSO Analysis
Item Analysis and Objectives
• Go to WVEIS private data site– Bottom left panel: Electronic CSR– Bottom left panel: WESTEST2/CSO Analysis
• Click on CSOs• See standard and number of students at each item
– What about after CIAS?• Think about the objectives themselves– Alignment
Item Analysis and Objectives
• Danger:– Objective-level sampling– Overemphasis on a single objective
• Consideration:– Consider learning progressions– Foundational objectives– Reporting (e-CIAS) is only the first step– What are the linked objectives and what tools
can help you analyze student mastery of those skills?