growth models
Post on 05-Feb-2016
35 Views
Preview:
DESCRIPTION
TRANSCRIPT
Including a detailed description of the Colorado Growth Model
1
OverviewReview Growth Model OptionsDetails on Colorado Growth ModelMajor technical questions
2
Basic DefinitionsStatus describes the academic performance
of a student or group (a collection of students) at a single point in time.
Growth describes the academic performance of a student or group (a collection of students) over two or more time points.
A growth model is a collection of definitions, calculations, and rules that quantifies student performance over two or more time points and supports interpretations about students, their classrooms, their educators, and their schools.
3
School-level “Growth” MeasuresYear to Year changes in percent met
safe harborGrade to Grade changes in percent met or
average scoresHow are my fifth graders doing compared to
my fourth gradersCohort growth over time in percent met or
average scoresHow has my 2009-10 third grade class
performed overtime
4
Growth ModelsA growth model is a collection of definitions,
calculations, and rules that quantifies student performance over two or more time points and supports interpretations about students, their classrooms, their educators, and their schools.
Most growth models follow individual student growth, and these are the subject of discussion today.
5
Common Growth ModelsGain ScoresTrajectoryCategoricalResidual GainProjectionStudent Growth PercentilesMultivariate
6
Key Questions Growth ModelsWhat interpretation does the growth model
support Formative? School accountability? Teacher evaluation?
Program Evaluation?What is the statistical foundation?
Gain-based, enhanced status, multivariateWhat are required data features?
Vertical scale? Articulated cut scores? Large datasets? Factors beyond test scores?
What interpretations does the model support? Does aggregation at the school level make sense?
Does the model include growth to standard?What are common misinterpretations and threats
to validity?7
Gain ScoresDefinition: Calculates the gain in a student’s
test score from year to year. Example: 5th grade math score – 4th grade math score
Requires: vertical scale.Strengths: very easy to calculate, very easy to
aggregatedWeaknesses: may not be very reliable, easier to
manipulate, poor vertical scale lessens value.School measure: average gainsOregon data: Gains vary by previous
performance levels, see next slide.8
Gain Scores – Oregon Data
9
Growth From Grade 5
-10
-5
0
5
10
15
20
25
200 210 220 230 240 250
Growth to Grade 6 Growth to Grade 7 Growth to Grade 8
Trajectory ModelDefinition: If a student continues on this path,
where will she be in the future (Oregon’s current model)
Requires: vertical scale useful, but not requiredStrengths: Simple to computeWeaknesses: assumes students make same
growth year after year, targets for student already meeting may be difficult to determine
School measure: percent of students “on track”Oregon data: see next slide.
10
Percent of students “On Track”
11
Categorical ModelDefinition: evaluates students transitioning
between performance levelsRequires: cut scores, benefits from articulated
cut scoresStrengths: useful when vertical scale not
available Weaknesses: coarse measure of student progressSchool measure: percent on track or weighted
index of students improving their performance levels
Oregon data: none12
Residual Gain ModelDefinition: student gains are measured against
“expected gains.” This is an “enhanced status model.”
Requires: state level data, model to predict expected gains, does NOT require vertical scaling
Strengths: value added measure, statistically robust - many statistics can be derived from the measure (median, standard deviation, school variability)
Weaknesses: does not directly measure growth, growth to standard is an add-on
School measure: average residual gains.Oregon data: gains are based on the mean gain,
conditioned on the prior test score. See next slide.
13
Residual Growth versus Status
14
Projection ModelDefinition: uses a regression model to predict future
scores. (Oregon’s federal growth model pilot submission was a hybrid of this model and a trajectory model)
Requires: linear regression model, state level data. Strengths: empirically grounded: uses historic data
to help predict current student’s performance, can use data from multiple subjects, supports growth to standard
Weaknesses: assumptions, such a linear growth, are often untenable.
School measure: average predicted score, percent of students “on track”
Oregon data: I’ve looked at versions of this, but don’t have pretty data to present.
15
Student Growth PercentilesDefinition: student growth compared to other
students with similar score histories, expressed as a percentile. It is an “enhanced status model”.
Requires: state data, large data sets, does NOT require vertical scale
Strengths: includes growth to standard algorithm, is value added
Weaknesses: complex modelSchool measure: median or mean growth percentile.Oregon data: see next slide and later in this
presentation.
16
Growth Percentiles in Oregon
17
The Colorado Growth ModelBasic AssumptionsSample DataAdequate GrowthMajor technical decisions
18
Quick DescriptionStudents with two consecutive years of test
scores are provided with a growth percentile.Student growth is compared to “academic
peers”, who are students with similar score histories.
A student growth percentile of 60 means the student “grew” as much or more than 60 percent of students with similar test score histories.
Various percentiles of growth can be projected into the future to determine “Adequate Growth Percentiles”
19
GlossaryAcademic Peers – students with the same or
similar test scores in the past.Student Growth Percentile (SGP) – the
growth of the student as compared to her academic peers
Adequate Growth Percentile (AGP) – the growth percentile that needs to be sustained in order for the student to meet or still meet standard in 3 years.
20
Growth Percentiles in Oregon
21
Sample SGP Output
22
Prior Grade
Current
Grade
Prior RIT
Current RIT
Growth
SGP AGP
5th 6th 195 200 5 16 80
5th 6th 209 208 -1 8 72
5th 6th 209 214 5 34 64
5th 6th 209 218 9 61 58
5th 6th 209 222 13 85 53
5th 6th 209 226 17 96 48
5th 6th 227 232 5 57 27
Student Growth ReportStudent reports can include the following:Student test historyCurrent Growth percentile – is the growth
low, average, or highScores that correspond to 1st,35th, 65th and
99th percentile (for comparison)Projections into the future for various growth
percentiles
23
Sample Student Report
24
Aggregations at School LevelMedian SGP by subgroup
This is the recommendation by the SGP author, though others recommend mean
Median AGP by subgroupThis is the recommendation of the SGP author,
though others recommend meanSubgroups Making Adequate Growth
Comparison of the median SGP and the median AGP.
Percent meeting AGPThis is not in the proposed school ratings model.
25
Draft School Level Growth Report
26
Median SGP Median AGP
Subject
Subgroup2011-
122012-
13Combine
d2011-
122012-
13Combine
d
MathAll
Students45 51 47 34 37 36
Math LEP 51 54 53 56 61 58
Reading
All Students
55 52 53 26 28 27
Reading
LEP 45 48 47 65 63 64
Web-based reporting
27
Now the ComplicationsSGPs -- The model can take into account
more than two years of data for each student
By using more than two years of data we may get a better estimate of the student’s growth.
See the next slide for examples
28
SGPs – Higher Order Growth
29
Current Grade
Back 2 years RIT
Prior RIT
Current RIT
Growth
SGP Adjustment
7th 225 231 6 47
7th 206 225 231 6 77 +30
7th 211 225 231 6 69 +22
7th 216 225 231 6 57 +10
7th 221 225 231 6 45 -2
7th 226 225 231 6 33 -14
7th 231 225 231 6 17 -30
This data show how 3 years of test scores can affect growth percentiles.
Focus and Priority SchoolsNew “rating” system that will identify focus
and priority schools. Growth is a very strong factor in the rating.
30
Draft Focus/Priority Identification Weighting
Elem/Middle
High
Achievement 25% 20%
Growth 50% 20%
Subgroup Growth 25% 10%
Graduation -- 35%
Subgroup Graduation
-- 15%
Details on the waiver: http://www.ode.state.or.us/go/nextgen
Details on priority, focus, and model schools: http://www.ode.state.or.us/search/page/?id=3742
Priority, Focus and Model School Detail Sheets: http://www.ode.state.or.us/search/page/?id=3749
Policy and Technical Manual: http://www.ode.state.or.us/wma/policy/accountability/nextgen2012/nextgenaccountabilitymanual2012.pdf
Jon WiensEmail: jon.wiens@state.or.us
Contact Information and Links
31
top related