1
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
A new resource for the
identification and utilization of
Electronic Learning Assessment Resources
Overview for CTAP Staff
September 21, 2005
Presented by
John Cradler
Ruthmary Cradler
Chris York
2
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
What is an ELAR?
• A technology application that simplifies delivery, aggregation, and disaggregation of assessment data • It provides information that allows teachers and administrators to make informed decisions and to prepare instructional plans based on student data
3
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
1. ConductAssessments
2. ObtainRelevant Data
3. AnalyzeData
4. DetermineConclusions
5. PlanInstruction
6. ImplementInstruction
An ELAR should assisteducators in conducting data-driven instructional planning
4
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
Why do we need an ELAR information service?
• Increased State and Federal instructional accountability requirements for reporting student outcomes.
• Increased emphasis on using student assessmentdata to inform district, school, and classroom instructional planning decisions.
• Technology applications or ELARs are rapidly emerging as a potential tool to enable data-driven decisions
• Educators need information on how to selectand use ELARs that meet their local needs
5
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
How did the ELAR information service evolve?
• California Legislature was interested in enabling educators to make more meaningful and effective use of State testing data
• The legislature recognized that technology has the potential to contribute to more effective use of student testing data to plan instruction
• State Senator Jack Scott authored SB 1384 which authorized development of criteria, a review process, and a strategy for informing educators about how to select and utilize ELARs
6
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
How is the ELAR project collaboratively implemented?
• CLRN: Provides for the online database results of ELARreviews, coordinates the reviews, and is the host SETS project for the ELAR information service.
• TICAL: Provided input on the design of the criteria and will assist in designing and conducting professional development and posting information relevant to ELARs on the TICAL website.
• TechSETS: Assists in the technical review of ELARS and provides input as an ELAR advising entity.
• Learning Resources: A contractor to CLRN who provides overall leadership and management for the design and implementation of the ELAR information resources.
7
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
Who are the other important collaborators?
• CTAP: The 11 CTAP Regions along with regional assessment staff will disseminate information and training on the use of the ELAR information resource.
• CDE Educational Technology Office: The contracting and overseeing agent for the ELAR component of CLRN is the CDE.
• CDE Office of Assessment: Played a major role in developing ELAR review criteria and is an active participant in the review of ELARs.
• ACSA and CUE: Both were supporters of SB 1384 and will be featuring conference workshops on how to use the ELAR information.
• CARET: The Center for Applied Research in Educational Technology provides research on the use of ELARs.
8
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
How were the Review Criteria Developed?
• Research was conducted to inform development of criteria.
• Draft ELAR Criteria were developed.
• ELAR Advisors and TICAL Cadre members provided Input.
• CDE Offices of Assessment and Technology provided Input.
• Draft criteria were field tested, revised, and finalized.
• CDE Approved the ELAR Review Criteria and related
guidelines by the legislated date of 12-31-04.
9
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
The ELAR Review Criteria:
Three categories of minimum requirements for all ELAR applications have been established to cover the range of types of ELARs:
Type I: Configured to import, process, and enable analysis of California State testing data.
Type II: Not yet configured for California but can be modified toprocess and analyze California testing data.
Type III: Does not use state testing data but can enable locally collected test and other data to inform local instructional planning.
All ELARs for each of the three types must meet eight general requirements ranging from having correct spelling to enabling instructional planning.
10
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
The ELAR Review Criteria (continued):
A. Sources of data: this can range from student grades to the CST
B. Relevant background: data that could have on impact on student achievement
C. Reporting & analysis of data: this ranges from the ELAR being able to enable analysis of data from the district to the classroom level, the same students over time, disaggregating by a variety of sub-groups, customization of reports for particular groups, charting of data, etc.
D. Standards to guide ELAR design: this includes attention to both content standards as well as technical standards
E. Customization of the ELAR: ranges from local modification of reporting to purchasing only parts of the ELAR needed
11
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
The ELAR Review Criteria (continued):
F. General or other features: this ranges from inclusion of instructional planning templates to links to research-based strategies to address achievement gaps identified by the ELAR
G. Support for using the ELAR: this addresses the availability of assistance from the publisher ranging from training to an online ‘help-desk’.
H. Technical requirements: this includes bandwidth, hardware, needed technical support, security requirements etc.
I. Development and research: a description of any research or evaluation conducted on the cost vs. the benefits or impact of the program on instructional planning
12
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
The ELAR Application and Review Process:
A. The publisher matches the ELAR features and to the ELAR criteria as the major component of the ELAR Application.
B. CLRN/ELAR staff arrange a review date, time and location.
C. The review process begins with a publisher presentation of the resource to the reviewers.
D. Reviewers try out the resource and test the features.
E. Reviewers evaluate the publishers match of the ELAR to the criteria and look for areas of agreement/disagreement.
F. When the review is completed the resource information is posted on the CLRN/ELAR website.
13
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
After the review, what is provided?
A. ELAR Product Profile: A profile of each of the ELARs that were reviewed describing the target users of the resource, technical requirements, and specific training needs to effectively use the resource.
B. Narrative Summary: A summary of the salient features of the program based on the results of the review as well as information on appropriate uses of the resource.
C. ELAR Consumer Report Matrix: A matrix allowing for quick and detailed comparison of features identified in the review as a major or minor focus of the ELAR.
D. ELAR Complete review: The detailed profile description and analysis of the ELAR including ratings and a narrative summary.
14
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
The ELAR Consumer Report Matrix:
G. Support for using the ELAR 1. Online ‘help-desk’ is included as part of the ELAR package. x w l l l w l 2. Telephone ‘help-desk’ is included as part of the ELAR package. w l l l l l l 3. Availability of tech support, hours, days, etc., is described. l l l l l l l 4. Training on how to use the ELAR is part of the package. l l l l l l l 5. Streaming video tutorials are available on using the ELAR. x w l x x x x 6. User prerequisite skills needed to use the ELAR are described. l w x x l x l 7. User prerequisite knowledge of assessments is described. x w x x x x l 8. Sample reports generated by the ELAR are provided with explanations. w w l w l l l 9. Information regarding cost(s) and process for obtaining ELAR upgrades
is described. l w l l l w l
10. User feedback on the ELAR is available. l l l l l l l 11. Resources for obtaining professional development for using the ELAR
are listed l l l l l w l
ELAR Title and Publisher Electronic Learning Assessment
Resources Review Criteria
Rating Key: Ratings reflect the Review Criteria focus for each ELAR reviewed as of August 2005.
Major Focus ……… l
Minor Focus ……… w
Not Applicable ……x
1. Aperture Suite; Intelligent
Solutionz, Inc.
2. Assessa; Eyecues, Inc.
3. Edusoft Assessment
System; Edusoft
4. Instructional Data Management
System; ETS Pulliam
5. Instructional Mgt. Solutions;
SchoolNet, Inc.
6. MEASURES Software Suite;
Datawise, Inc.
7. STARS; SchoolCity, Inc.
SampleELAR
ComparisonMatrix
15
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
From ELAR criteria development to ELAR utilization
4. Design &conduct ELAR
review training
1. Reviewresearch & existing
ELAR criteria
7. Maintain & Expand the
ELAR database
9. DisseminateELAR informationand training
10. Educatorsaccess, select & utilize ELARs
11. Evaluate ELAR Program development, implementation & impact & revise as needed
2. Establish &approve reviewCriteria (Dec 04)
3. Adapt CLRN& TICAL Websitesfor ELAR access
5. Publisherssubmit ELARs for review
8. Design ELARProfessionalDevelopment
6. Conductreviews of
ELARs
16
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
In summary, the ELAR information resource offers:
• A one-stop web-based location to review data management systems based on approved criteria.
• State-approved criteria enabling educators to compare/ evaluate their current instructional management systems.
• Research and information links on effective use of data management systems in instructional decision making.
• Professional development through TICAL and CTAP on how to use the ELAR website, to utilize ELARs to inform data-driven decisions, and apply the ELAR review criteria to guide local analysis and selection of ELARs.
17
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
What are the information features still under construction?
• As soon as there are sufficient ELARs posted an ELAR ‘search feature’ will allow users to match ELARs with their own profile as is now done with CLRN ELR searches.
• In collaboration with the Center for Applied Research in Educational Technology (CARET), ELAR related research findings will be linked to the ELAR information site.
• Online video streamed demonstrations of how to use ELARs is planned for the ELAR component of the CLRN site.
• User feedback will be collected for each of the ELARs posted and then summarized and edited for inclusion on the ELAR database.
18
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment
TechnologyInstructionELARELAR
ELAR project partner contact information:
Learning Resources
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
The ELAR Project is a service of the California Learning Resource Network (CLRN) funded by the CDE through a contract with the Stanislaus County Office of Education
Contact:Ellis Vance
Contact:Michael Simkins
Contact:John [email protected] Bob Beuthel
ContactBill Simpson:[email protected] [email protected]