Georgia's Online Options for Assessment: CRCT, EOCT, and the OAS

GAETC 2011November 2, 11:00-12:00

Joe BlessingMichael Huneke

Benefits of On-line Testing

• Ease of test administration• Security• Delivery of Accommodations• Preparation for future of testing

Test Administration

• Easier to administer without inconvenience and labor intensive demands of paper documents

• Students view one item at a time• Items are shown larger than on paper/pencil• Research shows comparability between online

and paper/pencil testing• Students motivated by online format

Research• CRCT comparability study showed equity in scores between on-line and

paper/pencil• Paek (2005) presented a table summarizing the results of recent comparability

studies. Adding up the number of grade/subject combinations, we see that out of 97 cases the results for 74 were deemed comparable, in 8 the computer-administered test appeared more difficult, and in 15 the paper test seemed more difficult. – Paek, P. (2005). Recent trends in comparability studies (PEM Research Report

05-05). Available from http://www.pearsonedmeasurement.com/downloads/research/RR_05_05.pdf

• Studies asked students who took computerized tests whether they would prefer to take future tests on computer or on paper. In all such studies located for this review, the majority of students indicated their preference to test on computer (Bridgeman, Lennon, & Jackenthal, 2001; Higgins, Russell, & Hoffman, 2005; Glassnapp, Poggio, Poggio, & Yang, 2005; Ito & Sykes, 2004; Johnson & Green, 2004; O’Malley et al., 2005; Richardson et al., 2002; Sim & Horton, 2004).

Security

• No paper/pencil tests in building/district• No answer documents handled • Read-aloud delivered by computer rather than

the test administrator• With paper/pencil, students can potentially

view multiple answers on another student’s answer sheet. Online, they can only see one question at a time and the answer only after it is chosen.

Security

• Online testing offers unique security issues. Test examiners should evaluate testing environment for security concerns.– Create seating arrangements that keep students

focused on their own computers.– Use monitor shields to maintain privacy. – Make certain that there are enough proctors in

the room to respond to technical issues and allow active monitoring.

Accommodations Accommodations available Online

Large Print Oral reading of test questions (Text Reader) Audio amplification device in conjunction with text reader Basic four-function calculator Untimed Test

Accommodations not available Online Braille Color overlays, templates, or place markers Oral reading of reading passages Low vision aids

All other accommodations are administered same as Paper/Pencil mode

Accommodations

• Many accommodations delivered by computer and assigned by test coordinator leads to more consistency in administration.

• Read-aloud accommodations delivered– Read-aloud is pre-assigned and available for student. – Reading of test items is standardized– Reading of content is personalized and student-directed

• Allows for individual student pacing

• Multiple accommodation groups possible in a single setting

CRCT Online Retest Updates

• CTB improvements• DOE changes that would need to be

considered to promote large-scale usage• Additional marketing

Updates to On-Line testing program

• Two-day report turnaround after closing of test sessions

• School test coordinator has rights to re-open session

• No more wait time after network disruptions• Allow for underlining/highlighting of text• Multiple sessions per day (already allowed)• Enhanced test examiner experience

Sample On-line Testing Schedules

Location Session Monday Tuesday Wednesday Thursday FridayLab 1 1 Reading Gr 3 Reading Gr 5 Math Gr 5 Makeup Makeup

Lab 1 2 Reading Gr 8 Math Gr 8 Makeup Makeup

Lab 2 1 Reading Gr 3 Reading Gr 5 Math Gr 5 Makeup Makeup

Lab 2 2 Reading Gr 8 Math Gr 8 Makeup Makeup

Test Features

One item displayed at a time

Passage shown at all times

Click on text to read aloud. Option for speed of delivery(Accommodation)

Return to review items

Test Features

Option Eliminator

Scroll to end of passage

Test Features

Test Features

Formula SheetGrade 8

Overview of Online Retest Roles

Roles and ResponsibilitiesTechnology Coordinator

CTB servers Student

computers

Customer proxy, firewall, virus scanning, web content filtering

Customer network

Student Workstations MS Windows based computers with 1.0 GHz CPU, with 512MB RAM and

200MB free space Mac computers with 1.8 GHz CPU, 512MB RAM and 200MB free space Linux workstations High speed internet connection, minimum 1.5 Mbps per 150 workstations

upload and download

Architecture Overview

Network Requirements

Bandwidth T1, high-speed cable, or high-speed DSL recommended 1.5 Mbps upload and download

Firewalls Port 80 and 443 must be open

Proxy Servers Dynamic pages are not to be cached Allow both HTTP and HTTPS traffic for *.ctb.com Prioritize traffic to and from *.ctb.com Users must have credentials if proxy authentication is enabled

Web Content Filtering

Administrator / Student Workstation Hardware and Memory

Processor 1.3 GHz Memory 512 MB Disk space 200 MB

Monitor Minimum 800x600 pixels

Operating System Windows 2000 SP4, or XP SP3, Vista SP1 Mac: OS 10.4.8, 10.5.4, or 10.6 Intel Linux: Linux Fedora Release 11 (Red Hat), OpenSUSE 11.1, or Ubuntu 9.0

Additional Software Adobe® Air® 1.5; Adobe® Reader® 7.0 or later Java Runtime Environment JRE 5.0, Update 16

Web Browser (specific for Administrator only) Microsoft Internet Explorer 6.0 or 7.0 Mozilla Firefox 2.0

Internet Access High-speed internet connection: Minimum 1.5 Mbps upload and download –T1, high-speed cable, or high-speed DSL

Peripherals Headphones for students with read-aloud accommodation

Administration Key Functions

Manage Users Manage Students Schedule Tests Print Tickets Monitor Status Mark Invalid/ Irregular Test

Print Tickets

Monitor Status

Monitor Status

Mark Invalid/Irregular

EOCT Online Testing

Overview

• End-of-Course Tests are administered via paper and pencil and/or online. – Paper/ pencil assessments are only available during

the winter, spring and summer administrations.– Online assessments are available for all main

administrations, mid-month administrations, and (beginning in Winter 2011) retest administrations.

Suggested Schedule

One-day Administration:• Distribute materials, Student Authorization Ticket,

Reading Directions Section 1 ........................10 min.• Test Administration – Section 2.....................45-60 min.• Break between sections.............................. ........5 min.• Test Administration – Section 3 ....................45-60 min.• Collect test materials.........................15 min.

Suggested ScheduleTwo-day Administration:

Day One• Distribute materials, Student Authorization Ticket, Reading Directions

Section 1 ........................10 min.• Test Administration – Section 2 ..................................................45-60 min.• Collect test materials ........................................................................15 min.

Day Two• Distribute Student Authorization Tickets.............................................5 min.• Test Administration – Section 3 ...................................................45-60 min.• Collect test materials.........................................................................15 min.

TestNav

• TestNav is Pearson’s proprietary test delivery application that is used to access the Internet and online tests. TestNav must be installed either on a file server or on each test delivery workstation.

• Test Delivery workstations are used to deliver tests to students. Students access online tests by means of TestNav. TestNav does not require a browser.

System Requirements

• The minimum and recommended hardware requirements for all TestNav test delivery workstations are listed below.

Hardware Requirements Windows Apple / Macintosh

Processor Pentium III 733 MHz (PIII/4) 1.3 GHz recommended)

G3 500 MHz (G4 800 MHz recommended)

Processor (for TestNav only) PIII 500 MHz (PIII 800 MHz recommended)

G4 500 MHz (G4 800 MHz recommended)

Memory 512 MB RAM 512 MB RAM (1 GB RAM recommended)

Devices Mouse/Pointing Devices Mouse/Pointing Devices

Audio Headphones/Speakers Headphones/Speakers

Screen Resolution 800 x 600 (1024 x 768 recommended)

800 x 600 (1024 x 768 recommended)

The Future of On-line Testing

Suggested Movement to Testing On-line

– 2011-2012 At least one school re-testing online per district

– 2012-2013 More than one school re-testing online in every district (dependent upon district’s size)

– 2013-2014 Every school that administers re-testing online in a district should have some online testing

www.georgiaoas.org

04/10/23

Kathy Cox,

State Superintendent of Scho

ols“We will

lead the

nation in

improving

student

achieveme

nt.”

34

Teacher Use of the OAS

• A web-based system that allows teachers to create, administer, and monitor assessments using the same kind of items that appear on the end-of-year assessments.

• All items and reports are aligned to the Georgia State Standards and easily linked to georgiastandards.org.

• A tool for preparing students for the end of year assessments.

• More importantly, a tool for formative assessment throughout the school year.

District Wide Implementation

• Allows administrators to implement large scale district-wide benchmark assessments using a unique bank of items

• Three sets of pre-built comprehensive benchmark assessments for GPS Reading/ELA, Math, and Science in grades 1-8

• Two sets of Framework Assessments for Reading/ELA, Math, and Science in grades 6-8

• One set of EOCT assessments

Benchmarks and Frameworks

To create a test using the Item Bank, select the Tests tab

Select Create a new test

Give the test a name,

a unique identifier,

and choose any other desired

options for the test, then click

Submit

Choose a subject for the test

Choose the grade level from which to choose the items

Choose a Domain

Choose a Standard (if available)

Choose an Element (if available)

From the list of available items, select (by checking the box to the left of the item) the items to include in the test

Once tests are created, they must be assigned to students. To assign a test, select the Tests tab.

Select Assign test(s) to student(s).

Students Take Test

Select the name of the report that you want to

view

New Level 3 Benchmark Reports

Kathy Cox,

State Superintendent of Scho

ols“We will

lead the

nation in

improving

student

achieveme

nt.”

56

Kathy Cox,

State Superintendent of Scho

ols“We will

lead the

nation in

improving

student

achieveme

nt.”

57

Kathy Cox,

State Superintendent of Scho

ols“We will

lead the

nation in

improving

student

achieveme

nt.”

58

A = An attempt at areaB = pi x radius x heightC = radius2 x heightD = pi x radius2 x height

M6M3 Students will determine the volume of fundamental solid figures (right rectangular prisms, cylinders, pyramids, and cones)

Assessment for Learning

• Identification of students needing support (screenings),

• ongoing progress monitoring (formative assessment),

• process for common assessments (creating as a team, review results as a team, make instructional decisions as a team),

• modification of instructional techniques based on data.

Tiered Approach to Formative Assessment

Teacher

Grade or ContentTeam

School

District

Qtly System Data Meeting

Qtly System Data Meeting

Monthly School Data

Meeting

Monthly School Data

Meeting

OAS SupportMichael Hunekemhuneke@doe.k12.ga.us404 232-1208

Support Hours and Contact Information • Website: http://www.georgiaoas.org • Live Remote Assistance is found on the Support page• Software Support Phone Number - (866) 215-6881 • Software Support Hours of Operation - Monday through Friday

7am -9pm EST • Support Email Address - gaoas_support@hmco.com

Assessment Support

CRCTJoe Blessing404 656-2589• jblessin@doe.k12.ga.us• http://www.gadoe.org/

ci_testing.aspx?PageReq=CI_TESTING_CRCT

CTB Help Desk• 866 282-2249• GeorgiaHelpDesk@CTB.com• www.ctb.com/gacrct

EOCTMelissa Faux• mfaux@doe.k12.ga.us• 404 656-5975• http://www.gadoe.org/

ci_testing.aspx?PageReq=CI_TESTING_EOCT

Pearson Customer Call Center • 888 705-9414 • georgia@support.pearson.co

m