early childhood outcomes center new tools in the tool box: what we need in the next generation of...
TRANSCRIPT
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
New Tools in the Tool Box: What We Need in the Next
Generation of Early Childhood
Assessments
New Tools in the Tool Box: What We Need in the Next
Generation of Early Childhood
Assessments
Kathy HebbelerECO at SRI International
Presented at the OSEP EC Conference, December 2008
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Today’s TopicsToday’s Topics
The new National Academy of Sciences Report: Early Childhood Assessment: Why, What, How
What should the next generation of assessment tools look like?
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Reason for the CommitteeReason for the Committee
Controversy over Head Start National Reporting System (NRS)
Congressional mandate “review and provide guidance on
appropriate outcomes and assessments for young children.”
2 key topics: Identification of key outcomes for children 0-5 Quality and purpose of different state-of-the
art techniques
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Organization of the ReportOrganization of the Report
Overview of purposes What to assess: Outcomes
And Quality of Environment How to assess:
Technical discussion of validity Issues related to minority, ELL, children
with disabilities Thinking systematically
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
2 Key Principles2 Key Principles
The purpose of the assessment should guide assessment decisions
Assessment should be conducted within a coherent system of medical, educational, and family support services, that promote optimal development for all children
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
PurposesPurposes
Determining an individual child’s level of functioning Individual-focused screening Community-focused screening Diagnostic testing Establishing readiness
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Purposes (Cont’d)Purposes (Cont’d)
Guiding intervention and instruction Evaluating the performance of a
program or society Program effectiveness Program impact Social benchmarking
Advancing knowledge of child development
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Guidelines from Multiple Guidelines from Multiple GroupsGroups
Assessment should benefit children Assessments should meet professional,
legal, ethnical, standards Assessments should be age-appropriate or
developmentally/individually appropriate Parents/family should be involved in the
assessment whenever possible Assessments should be linguistically and
culturally appropriate/responsive
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Guidelines from Multiple Guidelines from Multiple Groups (Cont’d)Groups (Cont’d)
Assessments should assess developmentally/educationally significant content
Assessment information should be gathered from familiar contexts (NEGP), realistic settings and situations (NAEYC), or be “authentic” (DEC).
Information should be gathered from multiple sources.
Screening should be linked to follow-up assessment.
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Recommendations Related to Recommendations Related to PurposePurpose
Make assessment purpose explicit and public
Assessment strategy must match purpose
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
[Which] Outcomes[Which] Outcomes
Why this domain? Evidence of consensus around
value Evidence of continuity within
domain over development or links to other domains
Evidence that it is a target for intervention and affected by the environment
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Outcome: Domains (NEGP)Outcome: Domains (NEGP)
1. Physical well-being and motor development
2. Socioemotional development3. Approaches to learning4. Language (and emerging
literacy)5. Cognitive skills (including
mathematics)
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Classification ChallengesClassification Challenges
Boundaries are artificial Researchers (and test
developers) use different classification schemes
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Recommendations related to Recommendations related to Domains and OutcomesDomains and Outcomes
Go beyond the standard domains of language and mathematics
Need support for better measures of socioemotional development
D-4 “For children with disabilities and specials needs, domains-based assessments may need to be replaced or supplemented with more functional approaches”
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Validity and ReliabilityValidity and Reliability
Weight of evidence Not whether an assessment is valid
and reliable Use accumulation of evidence to
judge whether the assessment is suitable for the task for which it is intended.
Evidence pertains to specific type of uses.
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Assessing All ChildrenAssessing All Children
Much more research needs to be conducted on the validity of tools with minority children, ELL, and children with disabilities.
Children with disabilities Construct irrelevant skills as a threat to validity
Interrelatedness of domains Functional outcomes Universal design Many tools used with children with disabilities
have not been validated for the purposes for which they are being used.
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Instrument Selection Instrument Selection RecommendationsRecommendations
Select assessment with acceptable levels of validity
Examine data produced with the assessment for validity for purposes for which the assessment is being used.
Test developers need to collect and make available evidence related to validity for minority, ELL, children with disabilities.
Be very cautious about reaching conclusions about a group not well represented in the validation sample.
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Situating Assessment in a Situating Assessment in a SystemSystem
EC System has A goal for children Strategies to achieve this goal Infrastructure supporting the goal
and the strategies Selects assessment to be
compatible with other elements of the system.
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Systems RecommendationsSystems Recommendations
EC assessment system must be part of a larger system with strong infrastructure to support early care and education.
Standards Assessment Reporting Professional development Opportunity to learn Inclusion Resources Monitoring and evaluation
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Systems RecommendationsSystems Recommendations
Coherence Horizontally coherent – curriculum,
instruction, assessment aligned with early learning standards
Vertically coherent – shared understanding of goals at all levels; consensus about purposes of assessment
Developmentally coherent – taking into account what is known about young children’s skills and understanding develop
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Research NeededResearch Needed
New conceptual frameworks Application of technical advances,
e.g., IRT Span birth to 6 or 7 Social emotional measures Capture children’s growth toward
being able to meaningfully participate in variety of everyday setting
Use of technology
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Challenges with Current Challenges with Current Assessments for All Assessments for All Children/Children with DisabilitiesChildren/Children with Disabilities
Domains-based When should assessment look at only one
domain? When should it look at the whole child? How functional are current tools?
Varies Need for tools to assess the three outcomes
How authentic? How real? How much is lost with structured requests? And
for which children?
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Challenges with Current Challenges with Current Assessments for All Assessments for All Children/Children with DisabilitiesChildren/Children with Disabilities
Tool structure E.g., Work Sampling
Need for specificity – sometimes.. E.g., AEPS; High Scope COR
Do we want predictive validity? Universal design principles
DRDP as the exception Adaptations and accommodations
Early Childhood Outcomes CenterEarly Childhood Outcomes Center
Challenges with Current Challenges with Current Assessments for All Assessments for All Children/Children with DisabilitiesChildren/Children with Disabilities
Diagnosis vs. intervention planning purpose Must these be separate tools? Are the things we want to know about children to
prove eligibility so different from what we want to know to develop an intervention plan?
AEPS in JEI, December 2008 Recognition and Response/RTI’s tools for
young children IGDI’s How do these fit with curriculum-based
assessment?