1

ABCTools and WinScan Update

Kenneth Barbour and ATEAM MembersAccountability Conference 2008

February 11-13, 2008

2

How We Do Business• Previously Senate Bill 991 Now G.S 147-33.72C

– Oversight of DPI projects moved from Information Resource Management Commission (IRMC) to State CIO

• Project Management Process– Roles (Project Manager, Functional System

Analyst, QA Analyst, Developers, Support)– Processes (Software requirements, testing,

documentation)

3

How We Do Business

• Formation of ATEAM– Part of IT– Members:

Ken Barbour, Jim Rhew, Kevin Skelton, George Stubblefield, Faye Atkinson, Julien Alhour, Rhonda Carlyle, Phyllis Price, and Lori Ivey

4

How We Do Business - ATEAMFebruary 1, 2007

- Accountability Services Management Team (ATEAM) was established under the guidance and leadership of Mike Veckenstedt, Deputy CIO/Chief Technology Officer for Technology and Information Services.

- Kenneth Barbour was selected as our Team Manager.

- Other staff members included Julien Alhour, Faye Atkinson, Rhonda Carlyle, Robert Johnson, Kevin Skelton and George Stubblefield.

5

How We Do Business - ATEAMFebruary 1, 2008

- Annette Murphy, Technology Services Business/Enterprise Program Manager, was named as our leader to guide us as we transition to a full IT shop.

6

Last Years Conference

Review of Top 2 Issues from the 2007 Accountability conference.

Start a new list for 2008

7

How We Do Business - ATEAM

• Internal Request Tracking System (RTS)

• GoTo Meetings

• Statewide Teleconferencing

• Collaboration with Accountability Division to determine the business rules and confirm the authoritative sources.

8

Top Issues in 2007

• At the 2007 State Accountability Conference Testing Coordinators were asked what the top 2 issues were for them.

• The group first listed the items below. After group discussion it was a unmans decision that none of the items were negotiable. All the items listed were mission critical. Ken Barbour agreed with the group and advised that the staff would prioritize the list according to how the items could be expiated.

9

Top Issues In 2007

• The group did pick two but it was only to comply with the question.

• In conclusion it was agreed that all of the things listed must be done soon.

10

List of Issues in 2007

• 1 Fix upload of alternative test into NCWISE and SIMS (this task in on schedule now to be completed soon)

• Init control tab – add s to school on build set of files for all schools

11

List from 2007 Continued

• Stand alone Process upload/down load for FREE Reduced Lunch, Grad. Cohort, EC, IPT (data fixed by authoritative sources instead of TCs)

• GMF file process back like it was in 2005 for student accountability standards.

• Add FRL, EC, LEP status back to Winscan• Winscan to virtually create a record for every

student and every test

12

2007 List Continued

• Report to show student on path to graduate in 2009 under new standards.

• Report that shows years from 8th grade (repeating 9th graders)

13

List from 2007 Continued

• ABC Tools for Schools (Menu with Audit, Student Profile, Precode, EOC cohort audit)

• Combine or Remove Tabs on Main Menu of ABCTools (remove calendar tab, combine Web Links tab with NCLB/ABCs tab)

• Ability to set accountability standards for EOG and EOC separately in WinScan

• Growth for new math, Eng 1

14

List From 2007 Continued

• Planning slide– Detailed documentation for End-Of-Year

(EOY) process with step-by-step checklist.

– Training using ABCTools and EOY processes.

– More documentation– More training – More feedback from the LEAs

15

List from 2007 Continued

• Grade 10 Audit

• WinScan Match process for 05/06 OCS FT files

• Kids who didn’t take OCS Eng 1 (no HSCT reading) (S Code?)

16

List from 2007 Continued

• Winscan file load into SQL

• Rename Reset Options button on Student Profile tab to Reset Collection Cycle

17

New Support Model• Helpdesk Expert Automation Tool

(HEAT) - a knowledge based system that will enable you to ask your questions and get the answers you need to perform better in your job.

• Pilots– Tyrana Battle/Paul Kinsey, Region 4– Barbara Collins/Pat Lynn, Region 6

• Timeline for Implementation

18

Nuthing is working

• Ken Barbour’s Cell Number• 919-218-1329

19

Accountability Configuration Control Board (CCB)• Purpose of Accountability CCB

– Oversight/governance– Formal process for user input and feedback

• Website– http://www.rep.dpi.state.nc.us/ccb/– links to documents

• Charter• Meeting agendas• List of members

20

Accountability Configuration Control Board (CCB) Members

Sharon Allen, Chair Hoke CountyDonna Clelland, Co-Chair Asheboro CityVickie Matthews Catawba CountyTamra Carter Columbus CountyCarolyn White Union CountyNancy Holmes Harnett CountyHannah Youngblood Johnston CountyPam Henderson-Schiffman Iredell-Statesville David Millush Carteret CountyGeneva Squire Northampton CountySarah Davis Jackson CountyBarbara Collins RAC RepresentativeDottie Heath Charter Representative

21

Proposed Change:• Change the running of ABC/AYP Reports from ABCTools on

local systems and DPI server to DPI server

22

Items of Interest• Provision for verification reports (ABCTools)

– ABCs– AYP– Disaggregated Reports

• Change in scheduling of NC WISE cross enrolled students. Home school vs cross enrolled. A SIMS solution for visiting students is being investigated.

• End of Year Processing (EOY) not finalized– SIMS– NCWISE

23

WINSCAN UPDATES WINSCAN UPDATES

WinScan32 and IndivRpt32George StubblefieldIT Support Specialist

gstubblefield@dpi.state.nc.us

23

24

WinScan32 2006-07

• Released Version 2.0.6.5e on 2/7/2008

• Includes:– Science grades 5 and 8– OCS Life Sciences– NC WISE Upload files for 2006-07– Removed R/U reason codes

25

WinScan32 2007-08

• Released Version 2.0.6.4c on 1/14/2008

• Files to scan and score OCS Math I, English I, and Life Skills Sci

• NC WISE changes

• Several fixes

26

Next WinScan32 Update• Scores for 5 new EOC subjects

– Run “Fix percentiles” to add

• EOC NCCLAS - Fall

• Improved “CSWrCopy” processing to do “Fix percentiles” and to sort files. Also check for duplicates.

• Match Process checks for duplicates

27

Near Future WinScan32

• IPT for Annual assessment

• Extend2 Reading field test

• Computer Skills Performance

• Strand Summary Reports for EOC– Mean percent correct vs. strand mean

28

Future Future WinScan32

• SIMS to NC WISE ID conversion process

• Edit files:– Scores corrected if necessary– Can manually add a record for an absent

student without scanning an answer sheet

• EOC - Fall vs. Spring in different data files– Lots of file extensions?

29

Future Future WinScan32

• Match Process:– More optional state course codes when

importing member files

– For Competency and Computer Skills -students that have exited without a diploma and are not in membership but being tested, add a match code like Summer EOC so not flagged as an error

30

NC WISE Upload• NC WISE Upload Status Viewer

https://www.rep.dpi.state.nc.us/app/wisesco/

• Test Upload User Guide and Power Point: https://www.rep.dpi.state.nc.us/RACS/

• Submit “SCO” files in: /home/download/LEAs/RAC?/###/SUBMIT_SCO ? = RAC number, ###=LEA or Charter Number

31

NC WISE Upload

• New upload process replaces the old “Front End” and “Back End” processes.

• All tests will be uploaded to NC WISE via this method

• Currently only available for 2006-07 and 2007-08 data

32

NC WISE vs. SIMS

• EOC – achievement levels, different codes

• IPT – stored by assessment grade (IP0K, IP01, .., IP12, standard scores, administration, superior

• Writing 4/7/10 – WRTSCORE vs. WRCOMMPS

• EOG grade 8 – no REPFINDI , MAPFINDI

• Science 5 and 8 are being added to NC WISE. Definitions already exist in SIMS

33

NC WISE vs. SIMS

• Submitted change request to add achievement levels for EOC, WRTSCORE, and IPT by grade to SIMS

• R/U reason codes do not work for Retests – use “day of testing. Regular = 01, Retest 1 = 10, Retest 2 = 20

• “N” reason code and NCWISE– Scores do not print on the transcript– 1/31/2008 “N” reason codes ignored– Change request to remove from NC WISE data

34

IndivRpt32

• Released Version 1.14a on 1/24/2008

• Print “multiple files”– One screen to select sort varialbes

• Online Computer Skills– `Strands

• EOC achievement level descriptors and state percentiles for Algebra I, English I, and Geometry.

35

Future IndivRpt32

• Strands for all EOC reports– 0 to 20– 10 is middle

• Other reports???Extend1, NCCLAS, Science

36

QUESTIONS

37

Software Updates – Overview2008 ABCTools – available nowhttps://www.rep.dpi.state.nc.us/abctools08/ - ability to run/update individual schools instead of entire LEA - new drop down menus - ability to add new processing options quickly - new ABCTOOLS SFTP upload process that doesn't require SSH

key generation

School List Verification webpage - available nowhttps://www.rep.dpi.state.nc.us/2008/app/schoollist_verification/ - check school details; Accountability will use for AYP/ABCs - check status of SNA changes for acceptance by Accountability - check audit log of SNA changes since start of school year

38

Software Updates – OverviewADB website - available nowhttps://www.rep.dpi.state.nc.us/adb/ - available now but still under development - convenient browsing of database table descriptions - convenient browsing of coding reference tables

Collection scoreboard - available nowhttps://www.rep.dpi.state.nc.us/2008/app/scoreboard/ - shows at a glance the existence of any files missing or in error within an entire collection. (Not including WinScan/test data) - Status Viewer supports a new 'not expected' status that TCs

can indicate for special cases in order to clear 'missing' errors

39

Software Updates - Overview

NCWISE Test Upload process - available nowhttps://www.rep.dpi.state.nc.us/app/wisesco/ - new website for tracking status and online error reports - handles all DPI-administered tests - combination of immediate feedback and next day turnaround - updates student records regardless of current school (data

follows the student)

School level web logins - available nowhttps://www.rep.dpi.state.nc.us/2008/app/lea/schoolpw.cgi - TCs at their discretion can create/distribute/revoke school level

logins for access to certain web applications such as the 2008 CGR Record Search

40

Software Updates - Overview

New EDS process replacing FRL - available now - server process no longer translates non-matching into best

match on the fly, which was counter to the interest of transparency and auditing

- server process now takes EDS data as submitted without ID modification

- to ease ID cleanup effort for TCs and/or Child Nutrition, a feedback file is automatically generated utilizing the best match logic, which can either be imported into CN database, or used to correct EDS file itself

41

Software Updates - Overview2008 Data collection process and status viewer - available nowhttps://www.rep.dpi.state.nc.us/2008/app/lea/status_viewer.cgi - all collections up through March are available - new collections are set to be activated in a timely manner - Fall and Spring EOC are planned to be listed separately - Regular and Retests are planned to be listed separately - new 'Not Expected' status available to support the "Scoreboard"

web pages

42

Software Updates – OverviewNew LEA wise ID conversion process - available now - 2008 CONVID collection - allows TCs to audit/correct ID conversion map from SIMS to

NCWISE - allows TCs to fix individual ID issues when dealing with ID

matching into historical data, such as for the CGR process - Changes in file format from last year to now include year/school

wildcarding

Local Options and Feeders collection (SchoolList utility) - available now

https://www.rep.dpi.state.nc.us/2008/app/schoollist/

2008 Cluster collection – available now

2008 TAS collection – available now

43

Software Updates - Overview2008 LEPFIXER/LEPAUDIT/PreCode process – available now - no major overhaul, but better error checking/feedback than last year

2008 CGR Record Search web page - not yet available

2008 CGREXIT/CGRAUDIT process - not yet available

New 2008 GRAD collection process - not yet available - GRAD data will be submitted to accountability via new online

graduate survey web page used by schools - exact end of year procedure not yet finalized

2008 Levels File - not yet available

44

Software Updates - Overview

2008 ABCTools local end of year/AYP_ABC processing - not yet available

2008 preliminary end of year AYP/ABC web reports at DPI - not yet available

2008 ID conversion process on current year files - not yet available

- For LEAs that convert during the 2008 school year, to catch up all data collections within the current year to use new IDs

- This will apply to all collections except WinScan, which will have its own local process and procedure

45

Software Updates - Overview

Testdata scoreboard - proposed - would show at a glance any WinScan files missing or in error by

type of test

Separation of Fall and Spring EOC file formats - proposed - enables DPI to better manage EOC data by treating them as two independent data collections - impact to TC is additional WinScan file extensions for spring

EOC

New 2008 EC data collection - proposed - all new EC data collection using CECAS as authoritative source

46

Software Updates - Overview

Automation of scheduled collections – proposed - ACCDATA, ACCTEST, Member/Pre-Code Files

New 2008 GCODE data collection - proposed - better quality control of internal data entry by DPI

New AYP/ABCs report status web page - proposed - allows TCs to check if AYP/ABCs reports are in process of

being updated, or in queue to be processed

47

Software Highlights

• The ABCTools loading of MYSQL has been improved. For example a FDF load that took 8 hours before now takes about 44 minutes.

• The plan is to automatically load MEMBER, ACCDATA, and ACCTEST collections for NC WISE systems.

• The ATEAM is currently planning to migrate ABCTools reports to a centralized web application this year. The application will be available for LEAs and schools.

• Data submissions due June 30, 2008

48

QUESTIONS