experiences and lessons learned using publishers’ online assessment systems for osep reporting...
TRANSCRIPT
Experiences and Lessons Learned Using Publishers’ Online Assessment Systems for OSEP Reporting
Presenters:Barbara Jackson – Munroe-Meyer Institute,
UNMCJanThelen, Nebraska Dept. of EducationRuth Littlefield, New Hampshire Dept. of
EducationAugust 27th, 2007
3
What is Results Matter in Nebraska?
A statewide child and family outcomes system Designed to improve programs and supports For all young children birth to five, including
Part C and Part B 619 Served by school districts, the Early
Development Network, and their partners Results Matter applies to all school-based
early childhood programs in Nebraska
Results Matter in NE
4
Results Matter in Nebraska Purpose…
improve experiences, learning, development and lives of young children (birth to age five) and their families
inform program practices demonstrate program effectiveness guide the development of local and state policies
and procedures provide data to demonstrate results
5
Results Matter in Nebraska calls for curriculum and child outcomes assessment that ...
is based on ongoing observation of children engaged in real activities, with people they know, in natural settings
reflects evidence-based practices engages families & primary care providers as active
participants integrates information gathered across settings is individualized to address each child’s unique ways of
learning informs decisions about day-to-day learning opportunities
for children reflects that development and learning are rooted in culture and supported by the family
Results Matter in NE
6
Implementation Process for Results Matter in Nebraska NECTAC state planning framework with state
management team in January 2005 State & Local Infrastructure Personnel Development Implementation/Practices Child/Family/Program Outcomes
Engage State Level Child Measurement Task Force on implementation process
Implemented six pilot sites in 2006 Statewide implementation in January 2007
Results Matter in NE
7
Results Matter in Nebraska
Three child measurement tools….
The High/Scope Child Observation Record (COR) The Creative Curriculum Developmental Continuum Assessment, Evaluation and Programming System (AEPS)
Use of publishers’ online systems for supporting documentation of requiredassessments
Results Matter in NE
8
NH Preschool Outcome Measurement System A statewide system to measure outcomes for
preschoolers with disabilities Designed to improve programs & supports For all preschool children with disabilities
with an IEP (effective November 1, 2006) May be used more broadly, at local
discretion
9
NH Preschool Outcome Measurement System GSEG Cornerstone grant for Part C & 619
Engage stakeholders Align Child Outcomes (Birth through 5) with
Early Learning Guidelines Consider how to address family outcomes Federal reporting AND Improving
outcomes Focus on progress, not comparison
10
Two Lead agencies Part C: 10 Area Agencies with 14 programs Part B 619: 177 School districts with multiple
programs Shared goal: improve results for children with
disabilities and their families Part C: COSF 619: Publishers web-based data systems
Behind the scenes conversion to OSEP reporting requirements
NH Preschool Outcome Measurement System
11
Three child measurement tools… Assessment, Evaluation, and Programming (AEPS®):
http://www.brookespublishing.com/store/books/bricker-aeps/osep.htm
Brigance® Inventory of Early Development II (IED-II): http://www.curriculumassociates.com
The Creative Curriculum® Developmental Continuum Assessment for Ages 3-5: http://www.creativecurriculum.net/
Allowed for flexibility Assessor preference Best fit for the child
NH Preschool Outcome Measurement System
13
NDE Administrative Structure
254 School Districts & 19 Regional Service Units
State License Subscriptions School Districts Head Start Programs
NDE memos
Authorization forms completed as part of enrollment process
NSSRS: NDE student identification number
Results Matter in NE
15
NDE Contracts and Costs
Jan 2006 NDE emails surveys to districts to collect: Information on # of children in school district early
childhood programs Selection of tool(s)
Fall 2006 developed contracts with three Publishers Brookes Publishing – AEPSi.com $11.00 Pearson Learning – Highscope.net $11.00 Teaching Strategies – Creativecurriulum.net $10.95
Fall 2007 Head Start Agencies contract with NDE to be part of the license
Results Matter in NE
16
NH Administrative Structure
177 School districts in 80+ School Administrative Units
NHDOE memos Statewide Preschool Technical Assistance
Network (PTAN) Professional development, TA and networking Immediate communication vehicle
17
NH Administrative Structure
NHSEIS (NH Special Education Information System)
Student Identification numbers Limited authority to view personal data Limited resources to fund the system High level of commitment to quality and
simplicity for districts
18
NH Contracts
Local Districts sign on with publishers No State-Level Umbrella agreements Districts negotiate directly with publishers
NHDOE: online access to reports for OSEP district and state level (# & %) run 3 reports and compile as needed
Districts access to all assessment data Inform instruction Demonstrate effective practices Promote discussion with families and other early
childhood partners
19
NH Cost
Survey Monkey: estimate number of children and users for each tool (for pricing & training)
State provided estimated children and or users to publishers
Paid for by local districts NHDOE worked with publishers to establish
best rate (not as good as umbrella agreement)
20
NH Cost
AEPS One-time state fee
Establish data reporting system & Create state account: $1750
Special rate to Beta Testers: $9/child Rate for other NH users: $11/child Complete AEPS®, Second Edition (Administration
Guide, Test for Birth to Three Years and Three to Six Years, Curriculum for Birth to Three Years, Curriculum for Three to Six Years) $239.00 with 10% discount when subscribe to AEPSi
21
NH Cost
Brigance IED II One Inventory per user $159.00 each One Developmental Record Book per student
$35.00/10-pack or $329.00/100-pack Web-based IED-II Management System
annual subscription $124.95 each user first year ($99 for subsequent years)
State account charged at same rate as local user ($124.95)
22
NH Cost
Creative Curriculum $11.75 per child No state fee No additional cost for tools/materials (unless
want hard copies)
23
NDE Customizing Needs
Funding Source-allows for differential children
Special Education Funding Early Intervention Part C (Birth-3) Early Childhood Sp Ed Part B (3-5) Not Applicable
State Funded EC Funding Source EC Grant Program Even Start Migrant Program Title I Not Applicable
Other Funding Source Head Start Early Head Start Not Applicable
Child Newly Verified (IFSP/IEP) after 8.1.06 Yes /No Added 10 Character ID #
Results Matter in NE
24
NH Customizing Needs
NH Head Start Collaboration office is working with NHDOE to better coordinate outcome measurement across systems
Need to consider Work Sampling (used by some NH Head Start)
Work Sampling: user enters assessment information the COSF data point is generated the user can adjust up or down
25
NH Customizing Needs
Each Publisher offers different benefits and challenges
Each has a representative dedicated to NH Areas for negotiation:
Tracking students who move within the state Online and phone TA for users Ease of federal reporting
Always ask the next question!
27
NH Training/Professional Development
Assessment Fair May 2006 Publishers paid us: they provided
demonstrations and resource PTAN
local networking phone and online TA Coordinated series of publisher trainings
Preschool Leadership Conference (Sept 07)
28
NH Supports and Resources
Statewide Preschool TA Network (PTAN) http://ptan.seresc.net/
Preschool Outcome Measurement Toolkit NHDOE Special Education Memos Links to ECO
Developing new FAQ
29
NDE Training/Professional Development Statewide Training Series on Assessment – Rolled
out January 2006 Publishers provided first training (January) State trainers provided at three regional sites
(Spring) Results Matter State Conference on Measurement –
Spring 2007 Regional Training Series on Early Childhood
Curriculum – Planned for Fall 2008 Developed collaboratively and supported through the
Early Childhood Training Center
Results Matter in NE
30
NDE Supports and Resources
http://www.nde.state.ne.us/ECH/ECH.html
Click on Results Matter
NDE Administrative Guidelines
NDE Assessment Fidelity Process
Results Matter in NE
31
Why implement a fidelity process?
Maximize the credibility of observational data collected for Results Matter
Reliable and valid data maximizes quality of information Consistency among providers in scoring
observations (intra- and inter-reliability) Documentation to support scoring (validity)
Quality information forms the basis for program planning
Results Matter in NE
32
Who does the plan affect?
All providers responsible for administering and scoring observational assessments for Results Matter Early Childhood Practitioners
Early Childhood Special Education Practitioners
Related Services Providers
Results Matter in NE
33
Levels of Expertise
Fidelity process is based on providers’ experience
Tier 1: Providers new to the assessment
Tier 2: Providers experienced with assessment
Tier 3: Experienced providers who can mentor others
Results Matter in NE
34
Five Components of the Fidelity Process
1. All providers must meet reliability standards by April 1, 2008
2. New provider orientation
3. Continued group monitoring
4. Individual monitoring
5. Ongoing mentoring of providers
Results Matter in NE
35
Sample Plan & District Level Strategies
Tier 1: Process for Providers New to the Assessment Tool Check all that are part of your plan and add additional activities as
needed. Approximate # of providers at this level: ____
Score state reliability tapes and achieve reliability standard
Shadow experienced provider, documenting observations and ratings, checking for reliability
Supervisor reviews samples of supporting documentation
Results Matter in NE
37
AEPSi
Near entry only reports now available OSEP near entry – near exit report available
as of Sept 2007 State and district level reports Districts access online information about their
children and programs
39
Creative Curriculum
OSEP Reports Child Progress Reports Development of 9 point scale to assess at-risk
children CreativeCurriculum.net + Special Needs
Accountability Pack (SNAPTM) and OSEPreports.net
District and State level reports Local school districts: all information on their
children/programs
41
High/Scope COR
Child Outcome Reports OSEP report – Data Downloaded to
High/Scope Foundation for analysis Report is in progress – available Sept 2007
Results Matter in NE
43
Publisher/State Challenges
Different stages of online development Different levels of experience Creating interface with state data systems
and processes Addressing children with significant delays Responding to OSEP timelines Emerging OSEP guidance