alternate assessments on alternate achievement standards student population jacqueline f. kearns,...
TRANSCRIPT
Alternate Alternate Assessments on Assessments on
Alternate Alternate Achievement Achievement
StandardsStandardsStudent PopulationStudent PopulationJacqueline F. Kearns, Ed.D.
Elizabeth Towles-Reeves, MS
OBSERVATION
INTERPRETATION
COGNITIONStudent PopulationAcademic contentTheory of Learning
Assessment SystemTest DevelopmentAdministration Scoring
ReportingAlignmentItem Analysis & DIF/BiasMeasurement errorScaling and Equating Standard Setting
VALIDITY EVALUATIONEmpirical evidence
Theory & logic (argument)Consequential features
The Assessment Triangle & Validity EvaluationMarion & Pellegrino (2006)
Cognition Vertex Validity Cognition Vertex Validity QuestionsQuestions
1)1) Is the assessment appropriate for Is the assessment appropriate for the students for whom it was the students for whom it was intended? intended?
2)2) Is the assessment being Is the assessment being administered to the appropriate administered to the appropriate students?students?
Both are important for the validity Both are important for the validity evaluationevaluation
More Different Than More Different Than AlikeAlike
SOURCE: Education Week analysis of data from the U.S. Department of Education, Office of Special Education Programs, Data Analysis System, 2002-03
Issues in Teaching/Assessing
Students in Alternate Assessments Varied levels of symbolic communication
Attention to salient features of stimuli Memory Limited motor response repertoire Generalization Self-Regulation Meta-cognition Skill Synthesis Sensory Deficits Special Health Care Needs
Kleinert, H., Browder, D., & Towles-Reeves, E. (2005). The assessment triangle and students with significant cognitive disabilities: Models of student cognition. National Alternate Assessment Center, Human Development Institute, University of Kentucky, Lexington. (PDF File)
Previous Data Previous Data 165 Students across 7 states165 Students across 7 states Extensive documentation through 111 item Extensive documentation through 111 item
inventoryinventory Findings suggest:Findings suggest:
64% routinely use verbal language64% routinely use verbal language 46% routinely understand pictures used to 46% routinely understand pictures used to
represent objectsrepresent objects 11% don’t understand pictures used to 11% don’t understand pictures used to
represent objects. represent objects. Almond & Bechard (2005) An In Depth Look at Almond & Bechard (2005) An In Depth Look at
students who take alternate assessments: students who take alternate assessments: What do we know. Colorado EAG.What do we know. Colorado EAG.
Learner Characteristics Learner Characteristics Demographic VariablesDemographic Variables
Learner Characteristics (all on a continuum of skills): Expressive Language Receptive Language Vision Hearing Motor Engagement Health Issues/Attendance Reading Mathematics Use of an Augmentative Communication System
(dichotomous variable)
MethodologyMethodology
Four partner states chose to participate States 1, 2, and 3:
gathered data in the administration process for their AA-AAS via scannable document (i.e., bubble-sheet)bubble-sheet)
State 4: gathered data using Zoomerang, an online
survey package. N= 7,075
States & LCI Response States & LCI Response RatesRates
StatStatee
GeograpGeographyhy
DemograpDemographichic
SampleSample
NNResponRespon
sese
RateRate
StatStatee
11
EasternEastern RuralRural
SuburbanSuburban 35953595 75%75%
StatStatee
22
North North EasternEastern
Urban-Urban-
SuburbanSuburban27932793 100%100%
StatStatee
33
EasternEastern UrbanUrban 468468 91%91%
StatStatee
44
WesternWestern RuralRural 219219 47%47%
Alternate Assessment Participation Rates : %
Total populationState 1
.959%
State 2
1.14%
State 3
.766%
State 4
.55%
“Most significant cognitive disabilities”
Expressive LanguageExpressive Language
Expressive Language
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
State 1 State 2 State 3 State 4
State
Perc
enta
ge
Presymbolic
Emerging Symbolic
Symbolic
Receptive LanguageReceptive LanguageReceptive Language
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
State 1 State 2 State 3 State 4
State
Perc
enta
ge
Uncertain response
Alerts to input
Requires cues
Follows directions
Use of Augmented Use of Augmented CommunicationCommunication
Number of Students not using ACS
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
State 1 State 2 State 3 State 4
State
Perc
enta
ge
Presymbolic
Emerging Symbolic
ReadingReading
Reading
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
State 1 State 2 State 3 State 4
State
Perc
enta
ge No awareness
Aware of text
Reads basic sight words
Basic understanding
Critical understanding
MathematicsMathematics
Mathematics
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
State 1 State 2 State 3 State 4
State
Perc
enta
ge
No awareness
Counts by rote to 5
1:1 correspondence
Does computational procedureswith or without a calculator
Applies computational procedures
Who are the Kids? Represent ~1% or less of the total assessed population All disability categories were represented but primarily 3 emerge,
Mental Retardation Multiple Disabilities Autism
Highly varied levels of expressive/receptive language use Most students in the population use symbolic communication Level of symbolic language distribution is similar across grade-bands Only about 50% of the pre and emerging symbolic language users
use ACS Pre-symbolic expressive language users are more likely to have
additional complex characteristics. Most of the population read basic sight words and solve simple math
problems with a calculator. Lack of skill progression in reading across grade bands (elementary,
middle & high) Skill progression apparent in mathematics across grade bands but
still small
Limitations
Only four state participants Small sample size Global items in reading and math Participation rates at 1% or less
Cognition Vertex: Cognition Vertex: Validity Evaluation Essential Validity Evaluation Essential
QuestionsQuestions Who is the population being assessed?Who is the population being assessed? How do we document and monitor the How do we document and monitor the
population?population? What do we know about how they learn (theory of What do we know about how they learn (theory of
learning) academic content?learning) academic content? What do our assessment results tell us about how What do our assessment results tell us about how
the population is learning academic content?the population is learning academic content? Are our data about the population and theory of Are our data about the population and theory of
learning learning consistentconsistent with student performances with student performances on the assessment?on the assessment?
If not, what assumptions are challenged?If not, what assumptions are challenged? What adjustments should be made?What adjustments should be made?
ParticipationParticipation Theory of LearningTheory of Learning Student PerformanceStudent Performance
References Agran, M., Fodor-Davis, Moore, & Martella, (1992). Effects of peer-delivered self-instructional
training on a lunch-making task for students with severe disabilities. Education and Training in Mental Retardation, 27, 230-240.
Billingsley, F., Gallucci, C., Peck, C., Schwartz, I., & Staub, D. (1996). "But those kids can't even do math: An alternative conceptualization of outcomes in special education. Special Education Leadership Review, 3 (1), 43-55.
Brown, L., Nisbet, J., Ford, A., Sweet, M., Shiraga, B., York, J., Loomis, R. (1983). The critical need for non-school instruction in educational programs for severely handicapped students. Journal of the Association of the Severely Handicapped. 8, 71-77.
CAST (2002). Fox, (1989). Stimulus Generalization of skills and persons with profound mental handicaps.
Education and Training in Mental Retardation, 24,219-299. Haring, N. (1988). Generalization for students with severe handicaps: Strategies and solutions.
Seattle, WA: University of Washington Press. Hughes, C. & Agran, M. (1993). Teaching persons with severe disabilities to use self-instruction in
community settings: An analysis of the applications. Journal of the Association for Persons with severe Handicaps, 18, 261-274.
Hughes, C., Hugo, K., & Blatt, J. (1996). Self-instructional intervention for teaching generalized problem-solving with a functional task sequence. American Journal of Mental Retardation, 100 565-579.
Westling, D., & Fox, L. (2004). Teaching Students with Severe Disabilities. Columbus: Pearson (Merrell).
Whitman, T. L. (1990). Self-regulation and mental retardation. American Journal on Mental Retardation, 94, 347-362.
Contact InformationContact Information
Jacqueline Kearns, Ed.D.Jacqueline Kearns, Ed.D. 1 Quality Street, Suite 7221 Quality Street, Suite 722 Lexington, Kentucky 40507Lexington, Kentucky 40507 859-257-7672 X 80243859-257-7672 X 80243 859-323-1838859-323-1838 [email protected]
Elizabeth Towles-Reeves, MSElizabeth Towles-Reeves, MS 1 Quality Street, Suite 7221 Quality Street, Suite 722 Lexington, Kentucky 40507Lexington, Kentucky 40507 859-257-7672 X 80255859-257-7672 X 80255 859-323-1838859-323-1838 [email protected]
www.naacpartners.org