Download - 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division
![Page 1: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/1.jpg)
2012–2013 Holistic Rating Training Requirements
Texas Education AgencyStudent Assessment Division
![Page 2: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/2.jpg)
Disclaimer
These slides have been prepared by the Student Assessment Division of the Texas Education Agency.
If any slide is amended or revised for local use, please remove the TEA footer at the bottom of the slide.
2TEA
![Page 3: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/3.jpg)
Topics
TELPAS Raters Credentials
Responsibilities
New & returning rater definitions
Holistic Rating Training Overview
Training requirements
Basic training, calibration, and supplemental support
3TEA
![Page 4: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/4.jpg)
Rater Credentials
Each teacher selected to rate an ELL must
1) have the student in class
2) be knowledgeable about the student’s ability to use English in instructional and informal settings
3) hold valid education credentials such as a teacher certificate or permit
4) be appropriately trained, as required by TEA
4TEA
![Page 5: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/5.jpg)
TELPAS Rater Responsibilities
A student’s TELPAS rater is the teacher designated by the district as the official rater of the student’s English language proficiency.
The student’s rater must rate the student in all domains for which the student is eligible. A student is not permitted to have one rater for some domains and another rater for other domains.
5TEA
![Page 6: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/6.jpg)
New and Returning Rater Definitions
for K–1 and 2–12
•The training webpage informs raters that districts may sometimes require a returning rater to complete new-rater training and to consult with their testing coordinator if they need clarification. 6TEA
![Page 7: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/7.jpg)
Holistic Rating Training
It is recommended that districts and campuses determine in the fall who their TELPAS raters will be in the spring.
A training flowchart is provided on the next slide.
7TEA
![Page 8: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/8.jpg)
Returning Raters
Online Basic Training Course
Online Calibration (Sets 1 and 2*)
If not calibrated: Supplemental Holistic Rating Training
Final Online Calibration (Set 3)
Spring TELPAS Administration Procedures Training
As a key part of this training, information from the TELPAS Manual for Raters and Test Administrators is reviewed to prepare raters to proceed with online holistic rating training
Fall ELPS-TELPAS Foundational (Awareness) Training
For teachers who will be trained as new TELPAS raters in the spring if they lack this foundation
Online Calibration(Sets 1 and 2*)
New Raters
*Set 2 required only if not successful on Set 1
8TEA
Training
![Page 9: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/9.jpg)
TELPAS Administration Procedures Training
As part of annual spring TELPAS administration procedures training, holistic rating training requirements are reviewed with raters, as well as information about how to access the online training and calibration components.
In addition, raters receive training on assessment procedures such as how to assemble writing collections, how to record students’ proficiency ratings, etc.
9TEA
![Page 10: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/10.jpg)
Two Types of Training
Online basic training courseThis course is for new raters. It provides instruction on using the PLD rubrics and gives raters practice rating students in each language domain. There are separate courses for K–1 and 2–12.
Online calibration This is for all raters. Raters use the PLDs to rate students in each language domain. Raters have three opportunities to calibrate on assigned grade cluster.
10TEA
![Page 11: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/11.jpg)
Grade Clusters of Training and Calibration
Raters must know their assigned grade cluster to select the appropriate online training.
Raters should consult with their campus testing coordinator if they are unsure of their assigned cluster.
Grade Clusters
Grades K–1Grade 2 Grades 3–5Grades 6–8Grades 9–12
11TEA
![Page 12: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/12.jpg)
Raters with Students in Multiple Grade Clusters within Grades 2–12
Raters are required to train and calibrate in the cluster in which they have the most ELLs.
As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s).
TEA 12
Example: A grade 2–5 ESL teacher has most of her ELLs in grade 3. She must complete training and calibration in grades 3–5. As a best practice, she should review the online practice activities for grade 2 to check her readiness to apply the rubrics appropriately.
![Page 13: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/13.jpg)
Raters with Students in Multiple Grade Clusters that Include Grades K–1
Because of differences in the instructional content and rating rubrics, these raters must complete training and calibration for K–1 and at least one other cluster in 2–12.
Raters with more than one additional cluster should train in the cluster in which they have the most ELLs.
As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s).
TEA 13
Example: A grade 1–3 ESL teacher has most of his ELLs in grade 2. He must complete training and calibration in grades K–1 and 2. As a best practice, he should review the online practice activities for grades 3–5 to check his readiness to apply the rubrics appropriately for his 3rd grade students.
![Page 14: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/14.jpg)
14
When and where do raters take the online training?
Depending on campus arrangements, raters may either complete training and calibration during school hours, after school, or on weekends.
The online training system allows raters to access the training from any computer that meets the minimum system requirements.
Campuses inform raters of the details during annual administration procedures training.
TEA
![Page 15: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/15.jpg)
Why is calibration necessary?
Calibration helps ensure that raters have adequate training, including ample practice and feedback, before they assess their students in the spring.
Calibration ensures that raters clear their heads and consider only the elements of student performance included in the PLDs.
Calibration supports assessment validity and reliability and is an important part of holistically scored assessment processes.
15TEA
![Page 16: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/16.jpg)
Preparing for Calibration Sets
New raters must complete the online basic training course before beginning calibration.
Returning raters have the option to review the online basic training course (which includes rating activities) before beginning calibration.
16TEA
![Page 17: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/17.jpg)
Other Things to Know About Calibration
Raters affirm online that they will keep the contents of the calibration sets secure and confidential.
Calibration activities are taken from a bank and randomized. Trainees will rate different sets of students.
Raters can work at their own pace, go back and review students, and change ratings as they work.
17TEA
![Page 18: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/18.jpg)
Raters can exit and return later to finish. They click a “submit” button when they are finished with a set.
After completing a calibration set, raters immediately see results. Results show the rating assigned by the rater and the correct rating.
Raters see annotations explaining the ratings. Raters should use the annotations to go back and review any incorrectly rated students.
Other Things to Know About Calibration
18TEA
![Page 19: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/19.jpg)
Is calibration a test?
No, it is a training method that ensures that raters have enough guidance, practice, and support to assess students consistently and accurately.
19TEA
![Page 20: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/20.jpg)
Will raters be able to refer to any resources during calibration
activities?
Yes, raters should use their rating rubrics (PLDs) and refer to, as needed, information from the:
online basic training course
holistic rating PowerPoints produced by TEA
TELPAS Manual for Raters and Test Administrators
Educator Guide to TELPAS20TEA
![Page 21: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/21.jpg)
How many students must be rated successfully?
To be successful, raters need to rate
students in their assigned grade
cluster with at least 70% accuracy.
21TEA
![Page 22: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/22.jpg)
Supplemental Holistic Rating Training
Raters not successful after sets 1 and 2 must receive supplemental training.
The rater will meet with a district-appointed supplemental support provider.
After the rater has received supplemental training, he or she will be able to access the third and final calibration set.
22TEA
![Page 23: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/23.jpg)
Recap of Calibration Process
There are 3 sets of 10 students. Raters who calibrate on set 1 are done. Raters who don’t calibrate on set 1 go on to
set 2. Raters who calibrate on set 2 are done.
Raters who don’t calibrate on set 2 receive supplemental training.
Raters attempt third and final calibration set. Raters who calibrate on set 3 are done.
23TEA
![Page 24: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/24.jpg)
What happens if a rater is unsuccessful in calibrating?
Two outcomes are possible for individuals who complete the calibration activities but are not successful:
The district may choose not to assign the individual to be a TELPAS rater.
If the individual is needed to serve as a rater, the district must implement rater support procedures to ensure that the rater’s students are evaluated consistent with the rating rubrics.
Individuals are not authorized to serve as raters unless they complete the required training components.
24TEA
![Page 25: 2012–2013 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division](https://reader033.vdocument.in/reader033/viewer/2022051215/56814577550346895db2497e/html5/thumbnails/25.jpg)
Raters who take the basic training course get a certificate from the online TrainingCenter after completing the course components.
Raters will receive a certificate of successful calibration when they calibrate.
Course and Calibration Certificates
25TEA