2011 CRCT Pre-Administration

WorkshopFebruary 2011

Webinar Etiquette• Please use the Audio Setup Wizard in the Tools Menu to configure and test your audio settings before the presentation begins.• To eliminate interference from background noise in your area, please leave the Talk Button (lower left of the Elluminate window) on mute if you are not speaking.• Due to the number of participants, we request that questions be submitted via Chat.

AGENDA

• Training Materials• CRCT Overview• Pre-Administration• Testing• Post Test Activities• Reporting• Contact Information• Questions

TRAINING MATERIALS

• “Train the Trainer Model” for system training

• The Test Coordinator’s Manual and Examiner’s Manual are posted on the Georgia Department of education website at:

http://tinyurl.com/gacrct • The Manuals and other training materials are also posted

on the CTB website at: http://www.ctb.com/gacrct

TRAINING MATERIALS• Posted to both the GaDOE and CTB websites:

– CRCT Pre-Administration PowerPoint presentation– CRCT Pre-Administration Webinar recording link– Test Coordinator’ Manual (TCM)– Grades 3 – 8 Test Examiner’s Manual (TEM)– Grade 3 Answer Document– Middle Grades Math Formula Sheet (DOE site only)– Read Aloud Guidelines & Tutorial (DOE site only)

• Additional materials posted to the CTB website:– Administration Documents

• Sample Packing List• Shipment Verification Form• Sample Security Checklist• Sample answer document• School Group List (SGL)• Group Information Sheet (GIS)• Stack Cover Cards

CRCT OVERVIEW

Will cover the following:• Administration Dates• Main Administration• Retest Administration• Test Construction

CRCT OVERVIEW: Administration Dates

• System testing dates were submitted in December/January through the Online Enrollment System.

– Main Administration: April 4 – May 6

• 9 consecutive day window – Retest Administration:

May 16 – July 29• 1 or 2 windows of 5 consecutive days each.

• 2 windows are available for different grade levels

CRCT OVERVIEW: Main Administration

• Main Administration: April 4 – May 6– Includes all students enrolled in grades 3 through 8,

except for those who take the GAA or CRCT-M– Due to budget constraints, the CRCT will not be administered in

grades one and two in spring 2011

• Make-up Testing– Make-up tests should be given within the system’s

scheduled 9 day administration window.– Outside of the system’s scheduled window, but within

the State Administration window, students in grades 3, 5, & 8 may make-up the Reading and Math sections.

CRCT OVERVIEW: Retest Administration

• Retest Administration: May 16 – July 29– Includes students who perform at Level 1 in Reading

(grades 3, 5, & 8) or Math (grades 5 & 8) during the Main Administration.

• Exception Testing– Qualified students who miss the system’s scheduled

Retest window may receive an exception to test within the state Retest Administration window.

– All requests for exception testing must be approved through the GaDOE.

• Online Retest Option: May 16 – July 29

Retest Administration: Online Retest Option• Administration Schedule

– May 16 – July 29, 2011• 5 day Reporting

– Systems testing in mixed mode (both Online and Paper/Pencil)• 5 day turnaround time (TAT) will begin at the receipt of

Paper/Pencil answer docs• System will receive reports for both modes together• Separate records for students taking different content in

different modes– Systems testing entirely in Online mode

• System must inform CTB customer service that entire system is online

• 5 day TAT will begin the day after the online test ends• For more information about testing online, visit the CTB-

hosted CRCT website: http://www.ctb.com/gacrct

Retest Administration:

Online Retest Option Important Dates

December 8, 2010 – January 21, 2011

System Test Coordinators Enrollment System CRCT 2011 Registration

December 8, 2010 – February 4, 2011

System Test Coordinators Enroll in CRCT 2011 Online Retest

February 17, 2011 System Test Coordinators Receive email confirmation of CRCT Online Retest participation

March 2, 2011 andMay 9, 2011

Technology Coordinators CRCT Online Retest Webinar Training

February 24, 2011 System Test Coordinators Additional Q&A Session for CRCT Online Retest

May 3, 2011 and May 10 , 2011

System Test Coordinators CRCT Online Retest training session

May 3, 2011 All Staff Receive CRCT Online Retest system accounts

Starting April 25, 2011 Technology Coordinators Complete CRCT Online Retest site readiness activitiesSubmit site certification email

May 16 – July 29, 2011

All Staff Administer CRCT 2011 Online Retest Sessions

CRCT OVERVIEW: Test Construction

• Content Areas– Spring grades 3 – 8: Reading, Math, ELA,

Science & Social Studies– Retest grade 3: Reading– Retest grades 5 & 8: Reading & Math

• All items are “Selected Response” (multiple choice)

PRE-ADMINISTRATION

Will cover the following:• System Test Coordinator Responsibilities• School Test Coordinator Responsibilities• Receiving Your Materials Shipment• Distribution to Schools• Security• Pre-ID Labels• Coding for GNETS

PRE-ADMINISTRATION: System Test Coordinator Responsibilities

• Reviews the Test Coordinator’s Manual (TCM) and Test Examiner’s Manuals (TEM).

• Organizes the training sessions for School Test Coordinators.• Organizes the training for assessment of students with

accommodations• Organizes system security plan.• Receives CRCT materials and verifies that all boxes have been

received.• Distributes CRCT materials and pre-ID labels to School Test

Coordinators. Please order additional materials as soon as possible.

PRE-ADMINISTRATION: School Test Coordinator Responsibilities

• Reviews the Test Coordinator’s Manual (TCM) and Test Examiner’s Manuals.

• Organizes training sessions for Test Examiners.• Organizes the training for assessment of students with

accommodations• Receives CRCT materials from the System Test Coordinator

and verifies that all boxes have been received.• Organizes school security plan.• Notifies the System Test Coordinator if additional materials are

needed.

PRE-ADMINISTRATION: Receiving Your Materials Shipment

• System Shipment includes:– System Packing List– School Packing Lists– Shipment Verification Form– System Security Checklist– Return Shipping Labels– System Overage materials (see next slide)

PRE-ADMINISTRATION: Overage Materials

• System Test Coordinators will use the System overage materials to provide additional materials to those school that exhaust the School Overage materials.– School Group Lists– Group Information Sheets

• Pre-coded and blank• Instructions on proper coding will be included.

– Test Coordinator’s & Test Examiner’s Manuals– Test booklets & answer sheets– Stack Cover Cards & paper bands

PRE-ADMINISTRATION: Overage Materials

• System selected their system-level overage from the Enrollment System.

• Schools receive 10% overage.• Schools also receive small group kits.

PRE-ADMINISTRATION:Receiving Your Materials Shipment

• When will I receive my shipment?– Shipping dates are estimated based on the dates provided through

the Online Enrollment System in December/January.– Test materials will be shipped to systems at least two weeks prior

to testing.– Shipping schedules will take into account Spring Breaks.

• Complete the System Inventory Checklist– Step-by-step instructions to be completed by the Test

Coordinators immediately upon receipt of materials.– System Inventory Checklist provided in the TCM on page 13.

PRE-ADMINISTRATION: Distribution to Schools

• School Test Coordinators – Complete the School Inventory Checklist provided in the TCM on page 15.

• Boxes are labeled for each School and include:– School Packing List & School Security Checklist– School Group Lists– Group Information Sheets and coding instructions– Test Coordinator’s Manuals & Test Examiner’s Manuals– Test booklets & answer sheets– Stack Cover Cards & paper bands

• Schools receive an overage of materials. – If additional materials are needed, School Test Coordinators contact the System

Test Coordinator.• Retain boxes for return shipping.

PRE-ADMINISTRATION: Security

• All Systems must adhere to the 9 day test window.• Materials kept in secure storage before, during, and after testing.• Student Test Booklets are presorted into class packs of 20 and 5 and must

remain in their shrink-wrapped packages as late as possible to avoid security concerns.

• School Test Coordinators use the School Security Checklist to check materials out to Test Examiners. All materials are returned to the School Test Coordinator for secure storage at the end of each day.

• Security wafer seals on all test books cannot be broken until testing begins.• Each classroom must have adequate staff to monitor testing.• Test Irregularities: Contact GaDOE for instruction. You may be asked to

complete the Test Irregularities form online on the MyGaDOE Portal.

PRE-ADMINISTRATION: Pre-ID Labels

• Pre-ID labels are included in each School Test Coordinator Kit.

• Three (3) barcode labels for each student.• Visible information includes:

– Student name & GTID – School and System names & codes– Grade– Student birthday & gender– Ethnicity, Race and PSRC– Barcode number

PRE-ADMINISTRATION: Pre-ID Labels

•Three barcode labels for each student•Do not use if inaccurate•Barcodes override bubbling•Additional labels can be used on test booklet

TCM

CRCT Answer document CRCT Test booklet Spare/CRCT-M

PRE-ADMINISTRATION: Pre-ID Labels• Pre-ID information is provided to CTB by GaDOE. If the

Pre-ID Label is incorrect:– Examiners must notify the School Test Coordinator– School Test Coordinator must update the school student records– Do not use incorrect Pre-ID labels.

• Pre-ID should be applied prior to testing.• Pre-ID labels should match student being tested EXACTLY.

Be aware of similar names. – Do not use the label if information is incorrect. Barcode

information will override bubbled information.• Barcode label may be used if the only inaccurate information is the

grade or teacher name.• If a student does not have a Pre-ID label, information

must be bubbled on the answer document.• When bubbling demographic information, fill in the text

boxes first, then bubble.

PRE-ADMINISTRATION: Pre-ID Labels• Barcode applied incorrectly

– If answers have been gridded:• Transcribe student answers & demographics onto a

replacement answer sheet.• Write VOID over the original booklet/answer sheet, and

return it with non-scorable materials.– If answers have not been gridded:

• Write VOID over the original answer sheet, and return it with non-scorable materials.

• Student transfers– Not applied to an answer document:

• Destroy the barcode.– Applied to an answer document:

• Write VOID over the original booklet/answer sheet, and return it with non-scorable materials.

PRE-ADMINISTRATION: Coding GNETS Programs

• In order to receive results for GNETS Programs, an identifier code needs to be bubbled on each student’s answer document in the box marked GNETS.

• Codes and instructions are included in the TCM

Alpine Program 01Burwell Program

02Cedarwood Program

03Coastal Academy Program

04Coastal GA Comprehensive Academy

05Mainstay

06Dekalb-Rockdale Program

07Elam Alexander Academy

08Flint Area Learning Center

09Harrell Learning Center

10H.A.V.E.N. Academy

11Heartland Academy

12

Horizon Academy 13Northstar Educational & Therapeutic Services 14North Metro Program 15Northwest Georgia Educational Program 16Oak Tree

17GNETS of Oconee 18Pathways Educational Program 19River Quest Program 20Rutland Academy 21Sand Hills Program 22South Metro Program 23Woodhall Program 24

TESTING

Will cover the following: • Preparation• Administration• Accommodations• Small Group Kits• Irregularities

TESTING: Preparation

• Remove or cover instructional materials that may prompt student responses.

• Arrange seating to promote individual work.• Post a sign on the outside of the classroom door: “Testing: Do

Not Disturb.”• One proctor is recommended for up to 30 students and required

for more than 30.• Examiner Materials:

– Examiner’s Manual– Test books (Gr. 3-8) & Answer sheets (Gr. 3-8)– Scratch paper for Math (no calculators)– Two No.2 pencils with erasers per student

TESTING: Administration

• Students take one content area per day.• All sections from a single content area must be

administered the same day.• Students can write in the test books.

Highlighters are not allowed.• Students take stretch breaks• Teacher monitoring– “Circulate & Check” to

prevent cheating, use of calculators, going ahead, or looking back.

TESTING: Accommodations

• Be familiar with accommodations listed in IEPs.• Accommodations must

– be required by the student in order to participate in the assessment

– be provided during routine instruction and assessment in the classroom

• Accommodations may not– alter, explain, simplify, paraphrase, or eliminate any test item,

reading passage, writing prompt, or choice option– Provide verbal or other clues or suggestions that hint at or give

away the correct response to the student• For Large Print and Braille, transcribe student answers on

to a regular answer document.

Coding Accommodations

• State Required Codes (SRC)• Accommodation Type (Setting, Presentation, Response,

Schedule)• Conditional Accommodation• Braille/Large Print• Read-Aloud Guidelines – in TEM

– Sample test items for Read-Aloud– http://tinyurl.com/crctreadaloud

Coding Accommodations

TESTING: Small Group Kits

• Small Group Kits are for those students who need a special administration of the exam in a small group environment.

• Small Group Kits consist of 10 test books of the same grade and form and the appropriate examiners manual for that form. Forms may be different than the forms shipped to schools.

• Small Group Kits are provided. Schools can also use regular overage materials to create Small Group Kits of a common form.

• Systems and Schools will receive a 10% overage. Additional Small Group Kits will be available from CTB Customer Service.

Irregularities (IR)

• Events and circumstances, or departures from standardized testing procedures are irregularities. They may have an impact on student performance that is not possible to define. They are reported and student scores are flagged simply to say, “There is something different about the conditions under which this score was obtained. Use caution in interpreting the score.”

TESTING: Irregularities

• All test administration irregularities must be reported by Examiners to the School Test Coordinator.

• The School Test Coordinator sends irregularities to System Test Coordinator for verification and to obtain the correct code.

• The System Test Coordinator contacts the GaDOE immediately for direction.

• If instructed to do so by the GaDOE, the System Test Coordinator will investigate and complete the Testing Irregularities Documentation Form provided online on the MyGaDOE Portal.

• Improperly coded irregularities may default to the holding of test scores

Some Examples of Irregularities (IR)

• Irregularities in Test Administration:• Materials being distributed inappropriately,• Directions not being followed,• Improper monitoring of test administration.

Invalidations (IV)

• Some irregularities are so serious in the way they affect student performances or compromise the integrity of the testing that they result in invalidation of student scores. • Content being disclosed, coached, or distributed, • Cheating,• Test Administrator altering responses during or

after testing.

Participation Invalidation (PIV)

• In addition to irregularities (IR) and invalidations (IV), systems will have to code for Participation Invalidations (PIV) for students who receive a accommodation that is not on the list of state-approved accommodations or not on the student’s IEP.

• Federal regulations stipulate that when inappropriate accommodations are used, students may not be considered participants in AYP calculations.

• In the CRCT, a potential PIV occurs when a student receives an unapproved conditional accommodation such as the reading of the test passages or use of the calculator on the Math test.

Coding Irregularities and Invalidations

Incorrect coding may result in no reports being sent

Reducing Irregular Test Administrations

• Ensure proper training of all test administrators• Maintain clear and consistent training and support

materials.• Make certain all accommodations are clear to test

administrators particularly if the Test Examiner is not the students’ regular teacher.

• Make certain that Test Examiners are constantly monitoring the testing environment.

• Respond to student actions appropriately and immediately.

• Implement a program for reporting irregularities during testing.

Characteristics of a Quality Investigation

• Examiner should notify Principal and School Test Coordinator of incident.• School Test Coordinator should notify System Test Coordinator.• Request detailed written statements from all parties involved if necessary.

– Follow up on details (connect the dots)• Follow district procedures regarding misconduct.• Provide a written summary of incident and investigation findings.

– Consult with GaDOE as needed.• System Test Coordinator is responsible for reporting to GaDOE.• Note:

– Report to GaDOE immediately, before investigation.– Final documentation should be entered into portal.

POST-TEST ACTIVITIES

Will cover the following:• Test Examiner Responsibilities• School Test Coordinator Responsibilities• System Test Coordinator Responsibilities• Packing Return Shipment• Return Shipment Pick-up• Returning Make-up Materials• Return Shipment Errors

POST-TEST ACTIVITIES: Test Examiner Responsibilities

Test Examiner shall:• Collect all test materials daily.• Review answer documents.

– Where applicable, verify that the identifying information is correctly gridded.

– If instructed by the System Test Coordinator, complete Local Option Coding (Section 15)

– Complete “For Teacher Use Only” (Section 10-14)– Stray marks, transcribing – monitored by certified educator

POST-TEST ACTIVITIES: Test Examiner Responsibilities

Scorable Materials

• Completed Group Information Sheet (GIS)

• Used Student Answer Sheets (Gr. 3-8)

Non-scorable Materials

• Used Student Test Booklets

• Large Print and Braille Test booklets

• Unused and Voided Test booklets and Answer sheets

• Scratch Paper• Examiner’s Manual

POST-TEST ACTIVITIES: Test Examiner Responsibilities

So that districts are prepared to address “non-participants” during AYP processing note:

• Collect by school and by subject… the names of each student who should have tested, but was a non-participant

• Districts will be asked to categorize their “non-participation” as one of the following four choices…Medical Emergency, Illness, Other, Unknown

• You may direct questions on the definition of “non-participant” and the above categories to your district’s GaDOE Accountability Specialist

POST-TEST ACTIVITIES: School Test Coordinator

• Verify that Test Examiners have returned all materials. – Materials for make-up testing administered outside of

the system’s scheduled window must be ordered by the System Test Coordinator.

• Inventory scorable and non-scorable materials separately• Complete the School Security Checklist and place in the first

box of Scorable material. • System Test Coordinators will need to make copies of any

pages that indicate missing or nonreturned materials and place these copies in the security checklist envelope for each school. System Coordinator’s will retain all original school security checklists.

• Verify that “VOID” is written across every voided Test booklet and Answer Sheet. Return voided documents with non-scorables.

POST-TEST ACTIVITIES:School Test Coordinator

• Destroy all scratch paper• Destroy all soiled documents (biohazard). Record as

soiled on the School Security Checklist.• Verify that each GIS is filled out correctly• Complete the School Group List (SGL)• Arrange all scorable documents with the student

name face-up.• Secure each classroom’s scorable materials with a

paper band using a Stack Cover Card for each bundle and a Group Information Sheet for each class. (Refer to the illustration on page 29 of the TCM).

POST-TEST ACTIVITIES: Packing Scorable Materials

Packing Scorable Materials:Band together by

classroom:[Top of Stack]

• Stack Cover Card– 1 per band

• Completed GIS– 1 per class

• 1 class of Student Answer Sheets (Gr. 3-8).[Bottom of Stack]

POST-TEST ACTIVITIES: School Test Coordinator

• Use the original shipping boxes for the return shipment. If you need additional boxes, notify the System Test Coordinator to order them through CTB’s Customer Service Team.

• Pack scorable and non-scorable materials separately.• All non-scorable materials must be returned to the System

Test Coordinator within one week of the last day of the system’s scheduled test window. In most systems, schools return scorable and non-scorable materials at the same time.

• Improper packaging may delay your score reports. Detailed instructions are provided in the “After Testing” section of the TCM.

POST-TEST ACTIVITIES: System Test Coordinator

Packing Return ShipmentSystem Test Coordinators prepare materials for pick-up.Reminders:• Do not consolidate materials from more than one school in the

same box.• Use one label per box.

– Use WHITE Return Shipping Labels for scorable materials.– Use Yellow Return Shipping Labels for non-scorable materials.– Note: You will not need any additional shipping labels to return your

materials.• Identify the sequence of boxes for each school on each shipping

label (example: Box 1 of 10).• Securely seal all boxes.

POST-TEST ACTIVITIES: Return Shipment Pick-up

• Assessment Distribution Services (ADS) will contact systems 5 business days before the scheduled pick-up to verify return shipments.– The pick-up date for scorable materials is the fifth

business day following the scheduled completion of testing unless noted in the CRCT Enrollment System.

– The pickup date for non-scorable materials is the tenth business day following the scheduled completion of testing unless noted in the CRCT Enrollment System.

POST-TEST ACTIVITIES: Returning Make-up Materials

• Make-up Testing– In most systems, make-up testing is accomplished

within the system’s scheduled main administration window. These materials should be returned with the main administration materials.

– For make-up testing outside of the system’s scheduled testing window, contact CTB for a Make-up Testing Kit.

• The Make-up Testing Kit will include instructions for returning these materials.

POST-TEST ACTIVITIES: Return Shipment Errors

Document preparation errors and packaging errors slow down the scoring process and can delay your score reports.

Examples include:• Document preparation errors

– Missing GIS– Unused/used materials packaged together– Schools packed together

• Document data errors– Xeroxed GIS– Student marks on timing tracks– Barcode incorrectly placed– Pre-ID placed over the wrong section

POST-TEST ACTIVITIES: Return Shipment Errors

• Packaging & Transportation Errors– Use of boxes other than the custom CRCT shipping

cartons– Boxes incorrectly numbered– All materials not available for pick-up as scheduled– Unlabeled pallets/boxes– Labels applied incorrectly– Self-scheduling using a different carrier

Noted Changes

• We continue to follow the federal guidelines to include both Ethnicity (Hispanic/Not Hispanic) and Race (American Indian, Asian, Black/African American, Native American/Other Pacific Islander, White).

• IR/IV/PIV reporting errors may result in not sending reports.

• Updated guidelines for read-aloud accommodations for grades 3-8 are in the Test Examiners Manual.

• This year system-level overage materials of 5%, 7%, and 10% may be selected.

CRCT Reports

• First set of electronic Test Results (GaDOE Portal, 5 day TAT after representative sample is met)– PDF files – Class Roster (with summary page)– PDF files – Performance Level 1 Roster– System Data file and Record Layout

• Paper reports (10 business days after receipt of electronic results)– Class Roster (with summary page)– Performance Level 1 Roster– Individual Student Report– Student Score Labels

Summary Reports

Paper Results (Delivered to Systems in Early July) Summary Report of All Student Populations

School, System, and State Performance Summary Report

Class, School, System, and State

Re-Test Reports

• Electronic Test Results (GaDOE Portal, 5 day TAT after reasonableness is checked)– PDF files – Class Roster (with summary page)– PDF files – Performance Level 1 Roster– System Data file

• Paper reports (10 business days after receipt of electronic results)– Class Roster (with summary page)– Performance Level 1 Roster– Individual Student Report– Student Score Labels

CONTACT INFORMATION:Ordering Materials

• When System overage materials have been exhausted, System Test Coordinators can order additional materials by contacting the CTB Customer Service Team.– Phone: (866) 282-2249– Fax: (866) 282-2251– E-mail: GeorgiaHelpDesk@ctb.com– Hours: 8:00am to 6:00pm (Eastern)

• Orders will be shipped using 2nd Day shipping unless expedited shipping is needed to deliver before the start of testing.

CONTACT INFORMATION: Shipping Information

• Materials Order Shipments– Your initial shipment of CRCT materials will be delivered by

CEVA Logistics. For information, contact:• CTB Customer Service

– Phone: (866) 282-2249– Fax: (866) 282-2251– E-mail: GeorgiaHelpDesk@ctb.com– Hours: 8:00am to 6:00pm (Eastern)

– Deliveries will be made at least 2 weeks prior to the first day of testing, taking spring break into account based on the dates provided to CTB through the Online Enrollment System.

CONTACT INFORMATION: Shipping Information

• Return Shipments– Assessment Distribution Service (ADS)

• Phone: (800) 963-7626• Email: info@ship-ADS.com• ADS will contact systems to schedule pick-ups.• The scorable materials pick-up date is 5 business days

from the last day of the scheduled test window.• The non-scorable materials pick-up date is 10 business

days from the last day of the scheduled test window.• The Pickup Schedule is provided in the TCM on page 37.

CONTACT INFORMATION: CTB Contacts

• CTB Customer Service Team.– Phone: (866) 282-2249– Fax: (866) 282-2251– E-mail: GeorgiaHelpDesk@ctb.com– Hours: 8:00am to 6:00pm (Eastern)

• Nadia Greer, Program Coordinator– Phone: (800) 538-9547, ext 7049– Fax: (888) 282-9579– E-mail: nadia_greer@ctb.com– Website: http://www.ctb.com/gacrct

• Dee Choudhary, Senior Program Manager– Phone: (847) 309-5957– E-mail: dee_choudhary@ctb.com

CONTACT INFORMATION: Georgia Department of Education

Joe Blessing, CRCT Program Manager- Phone: (404) 656-2589- E-mail: jblessin@doe.k12.ga.us

Test administration information– Phone: (800) 634-4106 or (404) 656-

2668

QUESTIONS