p20 coordinating council annual report 2010

8/6/2019 P20 Coordinating Council Annual Report 2010

1/32

O KLAHOMA P-20 D ATA COORDINATING COUNCIL

2010 A NNUAL R EPORT

Paul G. Risser, ChairPhil Berkenbile

Jon BrockStanley BryantRobert Buswell

Steven CrawfordSandy Garrett

Jill Geiger

Glen JohnsonSenator Clark Jolley

Donald ParkerAlex Pettit

Representative Earl Sears

January 24, 2011


2/32

EXECUTIVE S UMMARY Oklahoma P-20 Data Coordinating Council Annual Report, 2010

January 24, 2011

P-20 Data Coordinating Council The 2009 Oklahoma State Legislature enacted Senate Bill 222, the OklahomaEducational Accountability and Reform Act that established the Oklahoma P-20 DataCoordinating Council (OS Section 3-163 of Title 70)

to advise the State Department of Education, the State Regents for Higher Education, the Department of Career and Technology Education, the Office of

Accountability, the Oklahoma Employment Security Commission, theLegislature and the Governor on coordination of the creation of a unified,longitudinal student data system, a unified data system shall connect essential data elements relating to student level course data and coursegrades. The system shall facilitate the transfer of data across systems and among interested parties to address questions that cut across levels of theeducational system and agencies. The unified data system shall facilitate theaddition of data elements relating to testing, instruction and other

performance and demographic data. The unified data system shall beaccessible to a wide range of stakeholders and serve a variety of purposes,including improving teaching and learning, informing public policy, fosteringa culture of evidence-based decision making, conducting research,evaluating system and program effectiveness, and providing reports tovarious stakeholder groups.

The Council meets quarterly at the State Capitol and its deliberations are subject tothe Oklahoma Open Meetings Act (Title 25, O.S. 301-304) and the Oklahoma OpenRecords Act (Title 51 O.S. 24A.1-29). Membership and work of the P-20 DataCoordinating Council can be found on its website (OklahomaP-20Council.org).

Accomplishments of the P-20 Data Coordinating Council, 2010Among the activities of the P-20 Council in 2010 are the following:

clarified the requirements for Oklahomas student longitudinal data systemas specified by Senate Bill 222

compared the requirements of Senate Bill 222 to the current characteristicsand functions of the individual data systems of Higher Education, CareerTech,State Department of Education (K-12) and the Oklahoma EmploymentSecurity Commission

identified the expanded capabilities that are required to implement theOklahoma integrated statewide student longitudinal data system

surveyed n ationally recommended characteristics and functions of studentlongitudinal data system as defined by the Data Quality Campaign andAmerican Competes Act


3/32

examined student longitudinal data systems from other states

analyzed a representative set of questions that should be answerable byOklahomas statewide data system

specified the key obstacles to implementing Oklahomas student data system recommended actions for addressing obstacles to implementing an

integrated student longitudinal data system in Oklahoma

Obstacles and Actions The P-20 Data Coordinating Council identified five key obstacles to achieving afunctional integrated statewide student longitudinal data system in Oklahoma andhas stated the anticipated Council actions for 2011.

1. Student IdentifierA unique student identifier number is required to track studentsthroughout the data system, from pre-K through college and into theworkforce. Currently the K-12 student test identifier number is notconnected to the other education and workforce agencies, all of whichuse Social Security numbers.Action: Continue to explore the advisability and feasibility of usingSocial Security numbers; simultaneously define and implement astudent identifier matching system.

2. Electronic Transcripts The higher education unitized data system includes electronictranscript information; the K-12 data system does not have electronictranscripts or academic program achievement records for all students.Action: Agree upon the common information to be included onelectronic student records; insist that the K-12 system make electronicacademic records and transcripts available for all students.

3. FundingState appropriations have been allocated for some actions to buildstudent data bases in Oklahoma; however , no funding has beenlegislatively allocated specifically for building the statewide studentlongitudinal data system.Action: Insist the higher education, CareerTech and pre-K through 12systems, and the Oklahoma Employment Security Commission, jointlyfund and support Oklahomas integrated student longitudinal datasystem.

4. Information and Its Application The P-20 Council has identified example questions that should beanswerable by the student data system.Action: Offer a wide variety of stakeholders multiple opportunities todescribe how the statewide longitudinal data system can improvestudent learning and training from pre-K through college.


4/32

5. System Design and GovernanceOklahoma has not developed agreed-upon requirements and structuralcharacteristics of a federated student longitudinal data system, nor hasit decided upon the most effective governance and sharedresponsibilities for the student data system.Action: Define the structure and processes to be included inOklahomas federated longitudinal data system, describe thegovernance system and agency-specific responsibilities, and establisha Technical Advisory Committee to assist the P-20 Data CoordinatingCouncil.

TABLE OF CONTENTS

PAGEOklahoma P-20 Data Coordinating Council, Senate Bill 222 ... 1

Actions Required of the P-20 Council by Senate Bill 222 for 2009 .. 1

Requirements for Oklahomas Student Longitudinal Data System. 1

Relevant 2010 State Legislative Actions ....... 2

Actions of the Oklahoma P-20 Data Coordinating Council (2010)...... 3

A. Describe Oklahomas current student data system.. 3B. Identify national elements of statewide student longitudinaldata systems . 4C. Compare current Oklahoma data systems to nationalelements ... 5

D. Identify key obstacles to developing Oklahomas longitudinaldata system .. 7E. Propose solutions to the obstacles to the States longitudinaldata system ..... 8

Summary of Anticipated P-20 Council Actions in 2011..... 13


5/32

Appendix A 2009 Report of the P-20 Data Coordinating Council..... 15

Appendix B State of Minnesota Data Matching Process. 16

Appendix C Questions to be answered by the P-20 DataCoordinating Council ... 17

Appendix D Background for Defining the Data System Structureand Governance 22

Appendix E Technical Advisory Committee.. 23


6/32

OKLAHOMA P-20 DATA COORDINATING COUNCIL, SENATE BILL 222

The 2009 Oklahoma State Legislature enacted Senate Bill 222, the OklahomaEducational Accountability and Reform Act. Among other related actions, this Act

established the Oklahoma P-20 Data Coordinating Council (OS Section 3-163 of Title70)to advise the State Department of Education, the State Regents for Higher Education, the Department of Career and Technology Education, the Office of

Accountability, the Oklahoma Employment Security Commission, theLegislature and the Governor on coordination of the creation of a unified,longitudinal student data system to provide interoperability and efficient and effective storage, use and sharing of data among the State Department of Education, Oklahoma Department of Career and Technology Education,Oklahoma State Regents for Higher Education, Legislature, other

policymakers and executive agencies, and the general public.

Senate Bill 222 further stipulated that[F]or the purpose of this act, a unified data system shall connect essentialdata elements relating to student level course data and course grades. Thesystem shall facilitate the transfer of data across systems and amonginterested parties to address questions that cut across levels of theeducational system and agencies. The unified data system shall facilitate theaddition of data elements relating to testing, instruction and other

performance and demographic data. The unified data system shall beaccessible to a wide range of stakeholders and serve a variety of purposes,including improving teaching and learning, informing public policy, fosteringa culture of evidence-based decision making, conducting research,evaluating system and program effectiveness, and providing reports to

various stakeholder groups. The Council meets quarterly at the State Capitol and its deliberations are subject tothe Oklahoma Open Meetings Act (Title 25, O.S. 301-304) and the Oklahoma OpenRecords Act (Title 51 O.S. 24A.1-29). Membership and a record of the work of theP-20 Data Coordinating Council can be found on its website(OklahomaP-20Council.org).

ACTIONS REQUIRED OF THE P-20 COUNCIL BY SENATE BILL 222 FOR2009

Senate Bill 222 required that by December 31, 2009, the Chair of the P-20 DataCoordinating Council notify in writing the State Department of Education, the StateRegents for Higher Education, the Department of Career and Technology Education,and the Oklahoma Employment Security Commission of any actions necessary for each agency to take to enable the move to a unified data system and shallrecommend any statutory changes necessary to the Legislature and the Governor.The data system established shall include the elements described in section 6401

1 | P a g e


7/32

(e) (2) (D) of the America COMPETES Act (20 U. S. C. 9871 (e) (2) (D)). SeeAppendix A for this report, dated December 31, 2009.

REQUIREMENTS FOR OKLAHOMAS STUDENT LONGITUDINAL DATASYSTEM

The requirements of Oklahomas statewide student longitudinal data system(unified data system),specified by Senate Bill 222 and consistent with the America COMPETES Act, aresummarized as follows.

Purposes The Oklahoma statewide student longitudinal data system is to serve anumber of specific purposes, including to:

(a) improve teaching and learning(b) inform public policy(c) foster a culture of evidence-based decision making(d) conduct research(e) evaluate system and program effectiveness(f) provide reports to and be accessible to a wide range of stakeholders

Characteristics and FunctionsOklahomas statewide student longitudinal data system is required to exhibitseveral specific characteristics and functions:

1. include a unique statewide student identifier that does not permit astudent to be individually identified by users of the system2. provide student-level enrollment, demographic, and programparticipation information3. provide student-level information about the points at whichstudents exit, transfer in, transfer out, drop out, or complete P-16education programs4. ensure the capacity to communicate with higher education datasystems5. include a state data audit system for assessing data quality,validity, and reliability6. connect the essential data elements relating to student-levelcourse work and course grades

7. facilitate the transfer of data across systems and amonginterested parties to address questions that cut across levels of theeducational system and agencies8. facilitate the addition of data elements relating to testing,instruction and other performance and demographic data9. provide yearly test records of individual students with respect toassessments [American COMPETES Act section 6311(b)], enablinglinkages between and among student test scores, length of enrollment,and graduation records over time

2 | P a g e


8/32

10. provide information on students not tested by grade and subject11. include a teacher identifier system with the ability to matchteachers to students12. provide student-level transcript information, includinginformation on courses completed and grades earned13. provide student-level college readiness test scores14. for postsecondary education:

provide information regarding the extent to which studentstransition successfully from secondary school to postsecondaryeducation, including whether students enroll in remedialcourseworkinclude other information determined necessary to addressalignment and adequate preparation for success inpostsecondary education

RELEVANT STATE LEGISLATIVE ACTIONS (2010)

The 2010 Oklahoma State Legislature enacted several significant educationmeasures affecting the statewide student longitudinal data system and itsapplication to improving teaching and student learning.

Senate Bill 2033Senate Bill 2033 requires the State Board of Education to adopt the K-12Common Core Standards (released by the National Governors Associationand the Council of Chief State School Officers). The bill also created the

Teacher and Leader Evaluation (TLE) system, which requires a dramaticimprovement in the implementation of uniform, statewide student records.

Senate Bill 2211Senate Bill 2211 requires districts to report student data and test scores tothe State Department in order to maintain state funding.

Senate Bill 2330Senate Bill 2330 created a process and a set of models Oklahoma schools canuse to help implement real reform and subsequently be removed from thelow performance list. The bill allows a school to create an empowermentplan. If approved by the Districts Board of Education, administration and

teachers, the plan would impact the length of the school day or year,graduation policies, staffing, curriculum, assessment and other aspects of school operations.

House Bill 2747House Bill 2747 will help the Oklahoma Commission for Teacher Preparationbetter track the performance of new teachers.

3 | P a g e


9/32

ACTIONS OF THE P-2O DATA COORDINATING COUNCIL (2010)

The P-20 Coordinating Council has systematically focused on the expectations of Senate Bill 222 to:

1. describe Oklahomas current student data system,2. identify national elements of statewide student longitudinal data

systems,3. compare current Oklahoma data systems to national elements of

longitudinal data systems,4. identify key obstacles to developing Oklahomas statewide student

longitudinal data system, and5. propose solutions to the obstacles within the States student

longitudinal data system.

Oklahomas Current Student Data Systems

Education and workforce data systemsOklahoma has parts of a student longitudinal data system in place, but these piecesdo not yet form a complete functional data system in which information aboutstudents and teachers are connected through the P(pre-K)-20 continuum and intothe workforce.

State Regents for Higher Education The Oklahoma State Regents for Higher Education began a unitizedstudent data system (UDS) in 1977. This system, which includes manyvariables about student demography, enrollment and studentperformance, keeps track of students in all public higher educationinstitutions and some private colleges. Many studies and comparative

data are available on the website ( http://www.okhighered.org/studies-reports/ ).

Department of Career and Technology Education The CareerTech system tracks its students and the website, MeasuringUp , provides information and reports on several topics such asremediation rates, skill sets and standards(http://www.okcareertech.org/measuringup/index.html ).

State Department of Education The Wave, developed by the State Department of Education utilizingSchools Interoperability Framework (SIF) technology, is an electronicstatewide student information system currently used primarily forreporting. It contains information on student demography andenrollment, but has limited data on student performance(http://sde.state.ok.us/theWave/default.html ).

There are strong linkages between the Oklahoma Employment Security Commission(OESC) and both the Higher Education and CareerTech systems since students from

4 | P a g e
http://www.okhighered.org/studies-reports/http://www.okhighered.org/studies-reports/http://www.okcareertech.org/measuringup/index.htmlhttp://sde.state.ok.us/theWave/default.htmlhttp://www.okhighered.org/studies-reports/http://www.okhighered.org/studies-reports/http://www.okcareertech.org/measuringup/index.htmlhttp://sde.state.ok.us/theWave/default.html


10/32

both of these two systems can be tracked into the workforce. Because the StateDepartment of Educations Wave uses a separate student identification numberrather than Social Security numbers, students from the K-12 system are not linkedto the OESC workforce data.

National Elements of Longitudinal Student Data SystemsMost other states are intensely involved in building student longitudinal datasystems. These efforts are assisted by the increased ability to manage, analyze,interpret and display data and information in ways that may be understood andused by multiple stakeholders. In addition, states have invoked data securitysystems to protect data as appropriate while providing anonymous student-leveldata in ways that lead to improved teaching and student learning. Specifically, datacan only be made available to users and stakeholders to the extent allowed byfederal and state security and confidentiality requirements applicable to the data of each contributing agency (see http://www.dataqualitycampaign.org/resources/931 ),while strictly adhering to the requirements of the Family Educational Rights andPrivacy Act (FERPA).

The required characteristics of longitudinal data systems for improving instructionare now well recognized. Examples of these characteristics include ensuring thatthe data are rich enough to enable educators to triangulate on a particular problemwith confidence that different measures agree; that the data themselves aresufficiently fine-grained to be instructionally applicable (standards-level can behelpful, but item-level is sometimes preferable); and that the data are clean andaccurate. Practically, operation and response time of the system must be fast soteachers and principals can obtain analyses in time to make instructionaladjustments, with data that supports formative, summative, and predictiveassessments. Ultimately, the data system provides a number of values, includingbeing able to interpret how and what students really understand, for teachers andstudents to understand what is expected from individual standards, to see howstandards and individual items are taught, to share levels of student achievementwith inquiry teams, and to demonstrate that learning achievement can beincreased.

The Data Quality Campaign (DQC) is a national collaborative effort to encourageand support state policymakers to improve the availability and use of high-qualityeducation data to improve student achievement. The DQC has identified tenEssential Elements of a longitudinal data system and recommended state actions toensure effective use of these data.

1. Link state K-12 data systems with early learning, postsecondaryeducation, workforce, social services and other critical state agency datasystems.

2. Create stable, sustained support for a robust state longitudinal datasystem.

3. Develop governance structures to guide data collection, sharing and use.4. Build state data repositories (e.g., data warehouses) that integrate

student, staff, financial and facility data.5. Implement systems to provide all stakeholders timely access to the

information they need while protecting student privacy.

5 | P a g e
http://www.dataqualitycampaign.org/resources/931http://www.dataqualitycampaign.org/resources/931


11/32

6. Create progress reports with individual student data that provideinformation educators, parents and students can use to improve studentperformance.

7. Create reports that include longitudinal statistics on school systems andgroups of students to guide school district and state-level improvements.

8. Develop a purposeful research agenda and collaborate with universities,researchers and intermediary groups to examine the data for usefulinformation.

9. Implement policies and promote practices, including professionaldevelopment and credentialing, to ensure that educators know how toaccess, analyze and use data appropriately.

10. Promote strategies to raise awareness of available data and ensure thatall key stakeholders, including state policy makers, know how to access,analyze and use the information.

Compare Oklahomas Student Data Systems to National Elements

for Statewide Student Longitudinal Data SystemsSenate Bill 222 cites the America COMPETES Act that provides twelve elements thatare similar to the above Data Quality Campaign. Listed below are brief summariesof the America COMPETES ACT criteria and Oklahomas status on each one.

1. Assign a unique identifier for every student that does not permita student to be individually identified: The State Department of Educations Wave system assigns a ten-digit unique student identifier toall students who enroll in the public school system and these numbers areconcurrently used by the CareerTech system for shared students. TheHigher Education system uses Social Security numbers, as does the

CareerTech system and the Oklahoma Employment Security Commission(OESC). Higher Education, CareerTech and OESC now share dataroutinely. Higher Education and the State Department of Educationtransfer specific data on remediation. ACT and graduation rates areexchanged with Higher Education on a student-by-student basis but arereported on a school-by-school basis. ACT data are also re-aggregated andreported at district and state levels. However, currently, there are nocommon student identifiers that enable a complete P-20 studentlongitudinal database in Oklahoma.

2. Student-level enrollment, demographic characteristics and

program participation information: The Oklahoma State Departmentof Education collects student-level enrollment, demographic and programparticipation by students on a daily basis; the CareerTech and HigherEducation systems collect enrollment, demographic and programparticipation data at the end of each term.

3. Student-level information about the points at which students exit,transfer in, transfer out, drop out, or complete P-16 education

6 | P a g e


12/32

programs: The State Department of Education collects entry, exit, andtransfer data based on reporting from the districts according to the timeof the event; CareerTech collects student completion data after the closeof the school year on occupationally-specific programs. The HigherEducation system collects the required data for each term.

4. Capacity to communicate with higher education data systems: Asnoted above, data are routinely transferred between and among theOklahoma Employment Security Commission and the CareerTech andHigher Education systems. Data transfers with the State Department of Education are incomplete.

5. A state data audit system assessing data quality, validity, andreliability : The State Department of Education, CareerTech, and HigherEducation all have data quality checks built into their databases (as doesthe Oklahoma Employment Security Commission). The Wave currentlyexercises quality control using both SIF validation and additionalvalidation on all data objects and elements being received directly fromthe local educational agencies (LEAs) student information system. HigherEducation staff complete an electronic audit cycle, as well as an on-siteaudit for data submission at the colleges and universities. TheCareerTech system performs on-site audits at sites (schools, technologycenters, and prisons) on an as-needed basis.

6. Yearly test records of individual students with respect toassessments under section 111(b) of the ESEA (20 U.S.C 6311(b)): The Wave captures third- through eighth-grade math and Englishassessments as well as ninth- through 12 th -grade end-of-instruction scoresin its data system; however, the Wave does not capture electronictranscripts of all students. Higher Educations unified data system (UDS),which has been in place since 1977, includes electronic transcripts of allstudents and ACT scores. The CareerTech system does not own K-12students as these students are formally students of the State Departmentof Education where student records are kept.

7. Information on students not tested by grade and subject: TheState Department of Education provides information on students who arenot tested. Summary information is available on the Departmentswebsite.

8. A teacher identifier system with the ability to match teachers tostudents: The Oklahoma State Department of Education, Oklahoma StateDepartment of Career and Technology Education and the State Regentsfor Higher Education all include a teacher identifier system with the abilityto match teachers to students.

7 | P a g e


13/32

9. Student-level transcript information, including informationcourses completed and grades earned: Course completion andearned grade information is available from all three educational systems.For student-level data, the Higher Education system utilizes electronictranscripts and more than 300 high schools now have operational

electronic transcripts. The step yet to be completed is for all studentrecords and transcripts for all students in the states K-12 schools to beincorporated electronically.

10. Student-level college readiness test scores: Oklahoma collectsstudent-level SAT, ACT and Advanced Placement Exam data. HigherEducation and ACT, with the State Department of Education, haveconducted several analyses of college readiness and

cross-walked these assessment data with course curricula and with K-12state standards. This cross-walk analysis will need to be completed withthe newly adopted Common Core Standards.

11. Information regarding the extent to which students transitionsuccessfully from secondary school to postsecondary education,including whether students enroll in remedial coursework: Thislinkage is not complete because, as noted above, only some high schoolshave electronic transcripts and therefore convenient means to trackstudent-level data. Both the CareerTech and the Higher Educationsystems match data to determine remediation rates and the completion of post-secondary degrees. College enrollment data and remediation ratesare connected to individual high schools and are reported by both theState Regents for Higher Education and the Office of Accountability.

12. Other information determined necessary to address alignmentand adequate preparation for success in postsecondaryeducation: As noted above, most of this information is available fromthe Higher Education analyses with ACT comparing EXPLORE, PLAN andACT examination scores with college enrollment and remediation rates.Because the complete unit-level data are not yet included in thesecondary school data system, the student level alignment linkages arenot currently possible.

Key Obstacles to Developing Oklahomas Student Longitudinal DataSystem

The P-20 Data Coordination Council has identified five key obstacles to achieving afunctional statewide student longitudinal data system in Oklahoma.

6. Student Identifier The need to implement a unique student identifier number that willenable all students to be tracked throughout the data system, from

8 | P a g e


14/32

pre-K through college and into the workforce from high school,CareerTech programs or from colleges and universities.

7. Electronic TranscriptsAlthough the Higher Education unitized data system (UDS) recordstranscript information electronically, the K-12 system does notuniformly record transcripts or academic program achievementelectronically.

8. FundingState appropriations have been allocated for some actions to buildstudent databases in Oklahoma and these agencies have directedappropriated funds for this purpose; however, no funding has beenlegislatively allocated specifically for building an integrated, statewidestudent longitudinal data system to meet the requirements of SenateBill 222.

9. Information and Its ApplicationOklahoma has not sought to identify the questions that can or shouldbe addressable by a statewide student longitudinal data system, norhas the state systematically considered how such a system canimprove student learning in academic programs from pre-K throughcollege.

10.System Design and GovernanceOklahoma has not developed agreed-upon data system requirementsand structural characteristics, nor has it decided upon the mosteffective governance and shared responsibility for the statewidestudent longitudinal data system.

Solutions to the Obstacles to the States Longitudinal Data SystemOklahoma lags behind many other states in developing their statewide studentlongitudinal data systems. However, as evidenced by the Race to the Topapplications, there is considerable variability among the states in achieving thisgoal. The data below (latest but nearly a year old) show that as of March 2010,most states are making significant progress in linking data across the P-20education continuum and to the states workforce (Data Quality Campaign, 2010)).

Linkage Number of states with linkageEarly childhood and K-12 44

K-12 and postsecondary education 33Postsecondary education and workforce 29K-12 and workforce 10

Oklahoma has many of the required pieces of the longitudinal data system, theobstacles are known, the integration can be technically achieved, and the P-20Council has made progress on each of the identified obstacles and has identifiedproposed actions to address the key obstacles.

9 | P a g e


15/32

1. Student Identifier There are two techniques that can be used practically in Oklahoma to implement aunique identifier: Social Security Number [SSN] or state-assigned identifiers.

The Oklahoma Employment Security Commission (OESC) uses Social Securitynumbers for all of its workforce data analyses. Higher Education and CareerTechsystems use Social Security numbers, but the K-12 system uses a student identifiernumber generated with the Wave data system. Currently there is no connectionbetween K-12 and OESC and no consistent connection at the student level betweenK-12 and Higher Education. Since the CareerTech and K-12 systems share somestudents, there is the potential to connect those students between the two systemsusing the Wave-generated student identifier.

Using Social Security numbers has several advantages, namely, it is already usedby three of the systems in Oklahoma and is currently the only way to track studentsinto the workforce. Some states use the student's SSN as the unique identifier sincemost children are now assigned SSNs at a very early age. The federal governmenthandles the assignment and maintenance of the SSN system; thus, the burden of determining an identifier and assigning it is reduced. Parents and students canassist in providing the SSN when transferring schools and it is easier for them toremember the one number. When students move across state lines, those statesusing SSNs can easily verify records. Postsecondary application processes are alsosimplified when secondary schools can provide a SSN on transcripts. On the otherhand, some parents are uncomfortable providing their child's SSN and stronglyoppose its use. Individuals cannot be required to provide a SSN except for a fewpurposes such as federal loan eligibility or tax status. A very small number of students will not have a SSN.

Some states have created a pool of identifiers with the student identifier list

maintained by the state. Districts that receive a student transferring from anotherdistrict within the state can obtain the student's identifier through a locator systemat the state education agency. One advantage of this type of identifier is that thestate education agency controls the characteristics of the identifiers and can verifytheir validity. Uniqueness among student identifier numbers is assured. Further, theidentifiers can be kept separate from confidential data sources. A disadvantage of such a system is that school districts must rely upon the state education agency fortheir identifiers. Providing access to identifiers and ensuring the ability to assignthem at registration requires sound management.

P-20 Council ActionFor the reasons listed above, using Social Security numbers would offer the

greatest benefits to Oklahomas student longitudinal data system. Moreover,there is some indication that the federal government may favor thistechnique specifically for the purpose of these data systems. Thus, the P-20Council will continue to seek clarification on the possibility and advisability of SSNs for this limited purpose.

In parallel, the P-20 Council will investigate the possibility of adopting thestate-assigned identifier technique. The identifier would be a randomlyassigned number that bears no relationship to the personal or demographic

10 | P a g e


16/32

characteristics of the person it represents. The unique identifier would beassigned centrally using a batch matching and assignment that leveragesfields proven to be effective in matching records. These include but are notlimited to first name, last name, middle name, suffix, date of birth, genderand Social Security number. The fields used would be as broad as possiblewith few, if any required fields and Social Security numbers would not be arequired field. Minnesota uses a very similar system (Appendix B).

Either the P-20 Council or one of the primary stakeholders in the data systemwill accept responsibility for defining requirements within these parameters,surveying the marketplace and identifying a vendor for implementation.

2. Electronic Transcripts Transcripts or academic records for individual students contain most of the keyinformation about the students curriculum choices and learning achievement aswell as a record of the sequence of courses, grades, teacher names, and sometimesother records such as end-of-course test scores and additional assessments. Thisinformation, with the individual students identity properly protected, is essential fortracking and diagnosing learning achievements of students progressing throughOklahomas schools, CareerTech Centers, universities and colleges. Coursecompletion records and earned grade information are available from all threeeducational systems. For student-level data, the Higher Education system utilizeselectronic transcripts and thus this student-specific information can be readilyincorporated in the statewide longitudinal data system. Higher Education alsomaintains a student portal ( OKcollegestart.org) through which high school studentscan apply to college, and students beginning in the eighth grade can enroll inOklahomas Promise (OHLAP) program. Twenty-six CareerTech Centers alsoparticipate with electronic transcripts. This student portal has encouraged many(but not all) Oklahoma high schools to adopt electronic transcripts. However,incorporating electronic transcripts is yet to be completed for all student recordsand transcripts in the K-12 schools. Thus, the current obstacle is that not all primaryand secondary students have electronic transcripts and thus cannot benefit fromthe longitudinal data system.

P-20 Council ActionOklahomas statewide student longitudinal data system will not be functionaluntil the academic records of all students can be included electronically. Thisstudent-level information is necessary for designing assistance andinterventions to assure each student with the greatest chance of academicsuccess. Both Higher Education and CareerTech have electronic transcriptsand many high schools now provide students with the records. Thus, theobstacles are three: to ensure that all electronic transcripts contain at least aminimum set of information, to incorporate academic records of primary andmiddle school students, and to implement electronic transcripts in those highschools that have not already done so. The P-20 Council will define theinformation to be included in academic records and insist that the K-12system to complete electronic records for all of its students.

11 | P a g e


17/32

3. FundingOver the years, some state appropriations to Higher Education, CareerTech and theState Department of Education have been used to develop and support their datasystems. The State Legislature has also made specific allocations to the StateDepartment of Education to develop the Wave data system. Linking the existingdata systems will create much of Oklahomas statewide student longitudinal datasystem. No additional funding has been allocated to implement the requirements of Senate Bill 222 and there has been no coordinated process to assess the costs of meeting the mandates of the law.

P-20 Council Action The P-20 Council has reviewed several statewide student longitudinal datasystems from other states. In addition, though not successful, Oklahomasapplications to the U.S. Department of Education Statewide Longitudinal DataSystem (SLDS) grant, and the application to the National Center forEducational Statistics for the Race to the Top grant do provide someestimates of potential costs. The P-20 Council will insist that theimplementation of the integrated statewide student longitudinal data systemis a high priority for Oklahoma and that it must be jointly funded by theparticipating agencies.

4. Information and Its ApplicationAs clearly specified in SB 222, Oklahomas statewide student longitudinal datasystem will serve many purposes. Conceptually, it is useful to think about thelongitudinal student data sets at three levels:

Level 1 System DescriptionData at this level are useful for describing the specific district-, school- orcollege-level aspects of the educational system, such as graduation rates,remediation rates, numbers of students in classes, teachers of record andAPI numbers. This is the level of most of the current reporting from theeducation agencies and the Office of Accountability.

Level 2 Program AnalysesData at this level are used in various analyses, such as to evaluate theeffectiveness of programs, for example, which Common Core Standards arebeing met, the relationship between colleges of education and studentachievement, or the performance of students from various remedialprograms. Although some of these analyses (e.g., reports from HigherEducation, http://www.okhighered.org/studies-reports/ ) are available asindividualized studies, the current segmented data systems do not enablemany potentially useful longitudinal analyses.

Level 3 Student-Teacher SupportData at this level are used at the individual teacher and student level, forexample, in designing intervention for individual students, in providingdiagnostic information for parents, or determining which Common CoreStandards are taught best by individual teachers. Although a few schooldistricts (e.g., Western Heights in Oklahoma City) have made progress at

12 | P a g e
http://www.okhighered.org/studies-reports/http://www.okhighered.org/studies-reports/


18/32

the school level, without the longitudinal data system required by SB 222,Oklahoma cannot accomplish these goals on a statewide basis.

As discussed in previous sections of this report, the general requirements of statewide student longitudinal data systems have been well described by the DataQuality Campaign, the American COMPETES Act and Senate Bill 222. However,when designing the functional details and describing the most effective governancearrangements, other states have found it useful as part of their design processes toidentify the questions that should be answerable by the student data system. Thisprocess guides the type of information that should be incorporated in the datasystem and the array of data processing capabilities that will be required.

P-20 Council Action The P-20 Council undertook the task of identifying potential questions thatOklahomas longitudinal data system might be expected to answer. Thesequestions and a general description of the required data and their analysesare provided in Appendix C.

During the next three months, the P-20 Council will expand this opportunityto other stakeholders, including teachers, school principals, colleges of education, parents and other education stakeholders. The resultinginformation will help guide the design of the states student longitudinal datasystem.

5. System Design and GovernanceOklahomas statewide student longitudinal data system must meet the needs of many constituencies, for example, students, teachers, learning circles or othereducation-sharing collaborative groups, principals and superintendents, schoolboards, teacher and administrator unions, state lawmakers and policy-makersand the public. Thus, the design of the data system must incorporate theseneeds and present data and the results of analyses that are accessible andmeaningful for each of these constituencies. This means that the data systemwill provide data and information (synthesized data) in many formats andvisualizations, and it will include interactive tools for analyzing the results andpresenting them in interactive modes constructed to meet specific consumer(stakeholder) needs.

The design of Oklahomas statewide student longitudinal data system is acomplex task composed of several parts, including legal, economic, governance,technical/structural and operational issues. Fortunately, each of the fourexisting data systems (State Department of Education, CareerTech, HigherEducation and the Oklahoma Employment Security Commission) has experiencewith its current system. Several other states have mature P-20 data systems,and while few of them are complete, there are good examples available foremulation. Of particular interest is the way in which the data system is designedto meet the needs of multiple stakeholders, all of which have varied interests inthe resulting services and/or reports.

13 | P a g e


19/32

As described above, one of the goals of SB 222 is to put into operation anintegrated longitudinal database to ensure that policymaking at the state(legislative) level is based on the same information as that utilized throughoutthe pre-K 20 educational systems. Currently, data are available from multiplesites and users can pick and choose. For example, the Office of Accountability(http://www.schoolreportcard.org/contact.htm ) produces an extensive summaryof information ( Profiles, School Report Cards ) at the school, district, and statelevels. Higher Education and the CareerTech systems also produce reports,posting data and information on their websites as does the State Department of Education.

The Wave has significant interface capabilities. For example, in February 2010,a business intelligence module to support decision-making was implemented.

The Higher Education Unitized Data System (UDS) has interactive and datadisplay features; thus, collectively the Oklahoma education systems have astrong experiential foundation from which to work.

P-20 Council ActionBecause Oklahoma has not developed agreed-upon system requirements andstructural characteristics, nor has it decided upon the most effectivegovernance and shared responsibility for the statewide student data system,much work has yet to be accomplished. As an initial step, the P-20 Councilsought to achieve some basic agreements in preparation of addressing moredetails. Thus, the P-20 Data Coordinating Council has recommended that as abasic concept, the data system will be composed of linked databases and notbe built around a single, centralized data repository. With this model, dataare identified and presented from a central point, but are not actually moveduntil the data are needed. Thus, Higher Education, CareerTech, the StateDepartment of Education and the Oklahoma Employment SecurityCommission will each maintain its own data in its own system. Under thisconcept, these agencies (and others such as social service databases thatmight join in the future) will need to agree on technical protocols to physicallyshare data. At the central virtual data warehouse location (Data Mart inFigure 1), reports and data presentations will be created from data andanalyses provided as needed from agency databases.

Under this federated model (Figure 1) a single Data Mart will serve as thevirtual data warehouse and access point for information involving morethan one source.

14 | P a g e
http://www.schoolreportcard.org/contact.htmhttp://www.schoolreportcard.org/contact.htm


20/32

Career T

Figure 1. Example of the Data Mart concept for the Statewide Longitudinal DataSystem

Many of the details of the data system design are policy issues, such as dataownership, access and security, responsibility for analyses and review, andverification responsibilities. Other details are technical in nature, such asintegrating data bases as needed, maintaining data storage, creatingworkflows, constructing visualization tools and ensuring that both softwareand hardware enable policy decisions and can provide responsive productsfor the multiple stakeholders of Oklahomas student data system. All of thesepolicy and technical decisions will be made under the premise that data willbe made available to users and stakeholders to the extent allowed by federaland state security and confidentiality requirements applicable to the data of each contributing agency (seehttp://www.dataqualitycampaign.org/resources/931 ).

Over the next three months, the P-20 Council will examine policy andgovernance examples from other states and offer a draft description forOklahomas data system (Appendix D: Colorado). Simultaneously, the P-20Council will determine the necessary steps to appoint a Technical AdvisoryCommittee to provide guidance on the technical dimensions of the studentdata system.

SUMMARY OF ANTICIPATED P-20 COUNCIL ACTIONS IN 2011

15 | P a g e
http://www.dataqualitycampaign.org/resources/931http://www.dataqualitycampaign.org/resources/931


21/32

Based on its work over the last year, the P-20 Data Coordinating Council hasidentified its agenda for the year 2011.

Provide information and support The Council will respond to requests for information about the student longitudinaldata system, and provide support to and endorsement of appropriate proposals andprograms that will strengthen the data system.

Assess and report progress The Council will continue to monitor, assess and report on the states success increating its student longitudinal data system, including assessments referenced tothe requirements of Senate Bill 222, the ten criteria of the Data Quality Campaign,and the twelve criteria of the America COMPETES Act.

Student Identifier The P-20 Council will continue to seek clarification on the possibility and advisabilityof SSNs for this limited purpose. In parallel, the P-20 Council will investigate the

possibility of adopting the state assigned identifier technique. The identifier wouldbe a randomly assigned number assigned centrally using a batch matching andassignment that leverages fields proven to be effective in matching records, forexample, first name, last name, middle name, suffix, date of birth, gender andSocial Security number. The fields used would be as broad as possible with few, if any required fields; Social Security numbers would not be a required field.

Electronic TranscriptsOklahomas statewide student longitudinal data system will not be fully functionaluntil the academic records of all students are included electronically. This level of individual student-level information is necessary for designing assistance andinterventions to assure each student with the greatest chance of academic success.

Both Higher Education and CareerTech have electronic transcripts and many highschools now provide students with the records. Thus, the obstacles are three: toensure that all electronic transcripts contain at least a minimum set of information,to incorporate academic records of primary and middle school students, and toimplement electronic transcripts in those high schools that have not already doneso. The P-20 Council will compile the list of recommended information items forelectronic transcripts and academic records. The Council will clarify the scope of the problem in providing these electronic records in the K-12 system, and insist thateach student in Oklahoma schools has an electronic transcript or academic recordcontaining the recommended items.

Funding

Funding for the student data systems of Higher Education, CareerTech and theCommon Schools has been provided by state appropriations and grants fromseveral sources. The P-20 Council will insist that these agencies place a highpriority on the implementation of Oklahomas integrated, statewide studentlongitudinal data system.

Information and Its Application The real value of Oklahomas student longitudinal data system is the value itconfers to its stakeholders, including students, teachers, parents, principals and

16 | P a g e


22/32

superintendents, college administrators, legislators and other decision-makers andthe public. Oklahomas student data system will be most valuable if thesestakeholders have the opportunity to identify ways in which the data system canassist them. The P-20 Data Coordinating Council will assist in organizing multipleopportunities for stakeholders to share their expectations for Oklahomaslongitudinal student data system.

System Design and GovernanceOver the next three months, the P-20 Council will examine policy and governanceexamples from other states and offer a draft description for Oklahomas datasystem and the associated governance (Appendix D). Simultaneously, the P-20Council will determine the necessary steps to appoint a Technical AdvisoryCommittee to provide guidance on the technical dimensions of the student datasystem (Appendix E).

Appendix A. 2009 REPORT OF THE P-20 DATA COORDINATINGCOUNCIL

TO: Director Phil Berkenbile, Oklahoma Department of Career and TechnologyEducation

Director Jon Brock, Oklahoma Department of Employment SecurityCommission

Superintendent Sandy Garrett State Department of EducationChancellor Glen Johnson, Oklahoma State Regents for Higher Education

FROM: Paul G. Risser, Chair, P-20 Data Coordinating Council

DATE: December 21, 2009

SUBJECT: Senate Bill 222

Please recall that in 2009 the State Legislature passed Senate Bill 222, theEducational Accountability Reform Act. Among other provisions, the Act establishedthe P-20 Data Coordinating Council (OS Section 3-163 of Title 70). The Council ischarged with advising the State Department of Education, the State Regents for Higher Education, the Department of Career and technology Education, the Office of

Accountability, the Oklahoma Employment Security Commission, the Legislature,

and the Governor on coordination of the creation of a unified, longitudinal student data system to provide interoperability and efficient and effective storage, use and sharing of data among the State Department of Education, Oklahoma Department of Career and Technology Education, Oklahoma State Regents for Higher Education,Legislature, other policymakers and executive agencies, and the general public.

Specifically, the Act requires that by December 31, 2009, the Chair of the P-20 DataCoordinating Council shall notify in writing the State Department of Education, theState Regents for Higher Education, the Department of Career and technology

17 | P a g e


23/32

Education, and the Oklahoma Employment Security Commission of any actionsnecessary for each agency to take to enable the move to a unified data system and shall recommend any statutory changes necessary in to the Legislature and theGovernor. The data system established shall include the elements described insection 6401 (e) (2) (D) of the America COMPETES Act (20 U. S. C. 9871 (e) (2) (D)).

In fulfilling this responsibility, the P-20 Data Coordinating Council has identified thefollowing necessary actions for Oklahoma to create a unified student data system.

1. Maintain proper audit procedures to assure high standards of data qualityand

reliability.2. Ensure effective mechanisms to maintain confidentiality of student records

andadherence to the Family Education Rights and Privacy Act (FERPA)

requirements.3. Implement a student identifier number that can be used from pre-school

throughadults, connecting all three education agencies and the OklahomaEmployment Security Commission (OESC).

4. Create linkages between and among data systems so data can be transferredacross

systems and among interested parties to address questions that cut acrosslevels of the educational system and agencies.

5. Connect essential data elements relating to student-level course work andcourse

grades.6. Incorporate college-readiness measures into the data system.7. Provide help to and enable appropriate access to the unified longitudinal

database bya wide range of stakeholders to serve a variety of purposes, including

improving of teaching and learning, informing public policy, fostering a culture of

evidence-baseddecision making, conducting research, evaluation system and program

effectiveness,and providing reports to various stakeholder groups.

In addition to these required actions, the P-20 Data Coordinating Councilrecommends the following:

1. Incorporate teacher preparation attributes (e.g., certification type, school of origin)

into the data system.2. Incorporate analysis and business management tools into the system3. Implement greater interactive reporting capabilities to respond to a range of

stakeholders.4. Include student groups not now included (e.g., home-schooled) in the data

system

18 | P a g e


24/32

5. Complete basic policies such as data use/access protocols, data qualitystandards and

governance.

Your active participation as members of the P-20 Data Coordinating Council isappreciated as we all work collaboratively to strengthen Oklahomas student datasystem, thus enabling the strongest possible educational opportunities for ourstudents.

cc: Governor Brad HenrySenate President Pro Tempore Glen CoffeeHouse Speaker Chris Benge

Appendix B. STATE OF MINNESOTA DATA MATCHING PROCESS

The following description is taken directly from the State of Minnesota StatewideLongitudinal Education Data System (SLEDS) Charter (April 2010). This is theprocess used in Minnesota for matching student-level data from the MinnesotaDepartment of Education (MDE) and the Minnesota Office of Higher Education(OHE). The Minnesota Department of Education is parallel to the Oklahoma StateDepartment of Education and the Minnesota Office of Higher Education is parallel toOklahomas Higher Education system. There is no parallel to the OklahomaCareerTech system, but for reasons described above, CareerTech students can becaptured through Social Security numbers with Higher Education or with theOklahoma Employment Security Commission, or they are high school students

captured by the State Department of Education. The MARSS number below is theMinnesota Automated Reporting Student System (MARSS ) number, which is similarto the Student Testing Number produced by the Oklahoma State Department of Education through the K-12, Wave data system.

The [Minnesota] matching process maximizes data security, complies with federaland state regulation, and builds a system that can accomplish the intended research outcomes.

Construction of Data Files: Both MDE and OHE will appoint SLEDS InformationTechnology staff with access to full student record information from each agency to

construct:1) a reference table of personally identifiable data used only in the process tomatch MDE students with OHE students, and 2) data files used to populate the SLEDS System limited to the variablesidentified by the SLEDS Research and Data Advisory Committees as approved by the SLEDS Governance Committee.

Matching Student Records: Respective SLEDS Information Technology staff produce a reference table used to match students between OHE and MDE. These

19 | P a g e


25/32

reference tables will be exchanged between MDE and OHE using existing secure filetransfer protocols and technology. The reference table will include personally identifiable information that is stored in both agencies (e.g. first name, last name,date of birth, year of high school graduation, high school, and MARSS number).Probabilistic matching methodology will be used to determine how many and whichof the student records in both reference tables match exactly and the likelihood that the two records are the same student.

Assign Anonymous Identifier: Each student record in the reference table will beassigned a random anonymous identification number referred to as the SLEDSNumber. The SLEDS number shall be sent to the originating agency to be used to

populate the SLEDS data files. After the match process is complete the linked reference table data including all personally identifiable data shall be destroyed.

Appendix C. QUESTIONS TO BE ANSWERED BY THE P-20 STUDENTLONGITUDIAL DATA SYSTEM

Oklahomas P-20 Student Longitudinal Data System

P-20 Data Coordination CouncilNovember 22, 2010

Representative questions to be answered by Oklahomas statewide student longitudinal data system.Each question is subtended with an initial analysis about how the information in the data system couldbe used to answer the question.

A. Student Success

1. For a given school how are individual students performing on standardized tests atthe end of their final year at the feeding school?

o Select standardized test records of each student for each schoolo Classify schools by feeding schools and recipient schoolo Report average and distribution of scores for each feeding school and receiving school

2. How have students at a particular school progressed in each year of theirattendance?

o Design a value-added or progress model that defines progress achieved each year based on the students previous years performance compared to the current year and accounts for other variables such as socio-economic condition and schoolcharacteristics

o Track these data for each student in each schoolo Report progress by individual students (without identity) at each school by variables of

interest, e.g. school, grade level, curriculum, school district, teacher

3. How have students from a particular school fared in their first year of attendance atthe school subsequent to their departure? How have these students performedeach year until they complete college?

o Decide on the measure(s) of how students have fared for students, e.g.,standardized tests, assessments, progress model results, passing, grade point

o Identify students who transition from one school to the next each year for all studentso Calculate the track success measurements for each student for each year from

primary school through collegeo Report the results longitudinally from each school

20 | P a g e


26/32

4. What are the results of students enrolled in college remediation courses? Howmany complete their degrees verses non-remediation students?

o Record remediation records by course for each student o Record the curriculum and school pathways leading to the remediationo Report the remediation results according to each students preceding educational

pathway including courses, performance and school

5. How many OHLAP students require remediation? What percent of OHLAP studentscomplete their degrees?

o Record OHLAP in the record of each student beginning in the eighth gradeo Track each student into college and through graduationo Report according to middle school, high school, college and by course discipline

6. Achieving Classroom Excellence (ACE) and No Child Left Behind (NCLB) requiredhigh stakes testing. Are students succeeding at a greater rate in post-secondaryand post-graduate work as a result of these mandates?

o Define the high-stakes assessments to be used in the analyseso Track each student through K-12 schools into college and graduate programso Classify student records into homogeneous cohorts based on comparable conditions,

e.g. school conditions, socio-economic circumstances, teacher performanceo Compare student success over multiple years at each level on the basis of assessment scores with and without the ACE and NCLB high stakes testing

o Report results with proper caveats since this is not a true comparison of the samestudents with and without these tests

7. Can the P-20 data system relate completion and performance success of individualstudents in CareerTech programs to high school graduation rates, end of instructionrates, and performance in discipline-specific course sequences?

o Decide on performance measures for students in CareerTech programs

o Decide on definition of performance measures in discipline-specific course sequences

o Record graduation rates, end-of- instruction scores and performance in discipline-specific course sequences for each student

o Record completion and performance of each student in CareerTech programs

o Compare and report student success with CareerTech experiences on the basis of highschool graduation rates, end-of-instruction rates and performance in discipline-specificcourse sequences

8. Can the P-20 data system relate the college readiness of CTE students to ACT scores(English 18, Mathematics 22, Reading 21 and Science 25) and the AverageComposite ACT (Science, Math, Reading and English)?

o Identify CTE students and record their respective subject and composite ACT scoreso Track CTE students college admission success and of those entering college, track

including remediation, first year grade point average and persistenceo Compare ACT scores with measures of college readiness (e.g., admission, remediation,

persistence and grade point average in specific academic programs)o Report resulting relationship between ACT scores and college readiness on above

measures by post-secondary institution and CTE program

B. Student Transitions

21 | P a g e


27/32

1. Will a student who is starting to meet criteria as a likely drop out be identifiedindependently by the system once the criteria are met?

o Review the academic and school characteristics records of students who have dropped out over the past five years

o Classify the records by year of actual drop out o Build regression analyses of drop outs versus these dependent variables to identify

predictive criteria for the likelihood that a students will drop out o Calculate the predictive certainty of the regression modelo Apply these predictive criteria to current individual students to predict those

vulnerable to dropping outuse this analysis to create successful interventions

2. How many students drop out before completing CareerTech and Higher EducationPrograms, when do they drop out and what are the reasons given for dropping out?

o Review longitudinal database for each CareerTech program and college and university to calculate the number of individual student dropouts by school and year in school

o Address the question of why they drop out qualitatively by conducting surveys and interviews, and quantitatively by the regression analyses described in the previousquestion

3. Can the P-20 data system report whether a student with Cooperative Allianceearned credit continues on to post-secondary in the area in which he or shereceived alliance credit?

o Review the longitudinal database to identify students who earn credit in theCooperative Alliance

o Classify these individual students records by academic areao Track students by academic area prospectively into colleges and universitieso Report the results by Cooperative Alliance program, academic area and post-

secondary institution

C. Teacher Performance

1. For a given teacher, what was the performance of his or her students onstandardized testing in their prior year for the specific subjects to be assessed?

How much progress have these students made during their relationship with theteacher and how did they do in the subsequent year?o Decide on the assessment instruments to be used in the analyseso For each teacher, record the assessment scores of his or her students in the previous

year o Use the student progress model described in Question A.2 for value-added student

successo Conduct the same analysis for the subsequent year o Report the results for the current and subsequent year

2. Can the P-20 data system link a students performance (K-12 or post-secondary) tothe students teacher or professor, and then to that teachers or professors collegeof origin?

o Ensure that the longitudinal database includes the identity of the institution from

which the teacher received his or her education certification and similarly, theinstitution from which the professor received his or her terminal degreeo Decide on the measure of student performance in the K-12 system and in the college

or university o Use the longitudinal database to relate individual student performance to the teachers

or professors institution of origin

3. Does the system provide proper tools for assessment of educational professionalsand support personnel and their preparation/professional development successrates?

22 | P a g e


28/32

o Determine the assessments to be used in evaluating professional development of educational professionals and support personnel

o Ensure that the database includes these assessment results as well as professionaldevelopment opportunities

o Quantitatively compare assessments of professional educator and support personnelsuccess rates to participation in professional development opportunities

o For evaluating preparation, use the student progress model described in Question A.2

D. Curriculum

1. For a given curriculum how were students performing on standardized testing intheir prior year for the specific curriculum to be assessed? How much progresshave they made during their time with that curriculum and how did they do in thesubsequent year?

o Decide how to characterize each curriculumo Decide on the standardized tests to be used for evaluating performance oneach curriculumo Ensure that these curriculum descriptors and test scores are in the longitudinaldatabaseo For each student by curriculum, calculate the student progress as described in

Question A.2o Report the results for the current year and compare to the same results in thesubsequent year

E. Data System Function

1. Does the system adequately provide real-time diagnostics for parents, students andeducators to better prepare a student academically?

o For each educational level (e.g., primary, middle and high school, CareerTech,community college and university) decide on the measures of academic success (e.g.,standardized test scores, persistence, choice of curriculum, student progress modelresults)o Decide on the timeframe in which student performance assessment must be

available for teachers to enable them to create successful student-specific learningenvironmentso Decide upon how the results should be conveyed to parents, students and educatorso Decide on the definition of adequacy to be invoked in these continuousanalyseso Report the results to the three stakeholder groups

2. Does Oklahoma's P-20 data system track all participants in the common education,higher education and CareerTech systems without having three separate andindependent systems?

o The system will have four databases, but will be linked so that data can beshared for analyses and reported through a common portal

3. Are students in private systems of education being included in the system to theutmost way possible for transfer of information?o Some data from private colleges is included in the higher education database

4. Throughout the sequence of an individual student who moves from enrollment inhigh school to a technology center to a college, can the P-20 data system keep track of (a) the transition from system to system, (b) plan or program of study, (c)remediation, (d) retention, (e) assessment measures of performance, (f)completion, and (g) graduation in a reasonable time frame?

23 | P a g e


29/32

o Ensure data from each of the three systems (high schools, technology centers and colleges) are in the longitudinal student data system

o Track individual student records to include, transitions from education/training units,education/training plan or program, remediation, retention, test and assessment scores, grade/program completion credentials, and graduation milestones

o Report student numbers and distributions for each of the requested indications of student status

5. In evaluating the completion and performance success of individual students, canthe P-20 data system report if the student takes upper-level science andmathematics courses?

o Decide on the performance metric (e.g., test scores, results of student progressmodel as in Question A.2)

o Record whether each student has taken upper-level science and mathematics courses

o Record performance metric and completion steps for each student

o Compare and report performance and completion rates between those students whotook upper-level science and mathematics courses with those who did not

6. In evaluating the completion and performance success of individual students, canthe P-20 data system report if the student takes academic courses at a TechnologyCenter or Comprehensive High School?

o Decide on the performance metric (e.g., test scores, results of student progressmodel as in Question A.2)

o Record whether each student has taken academic courses at a technology center or acomprehensive high school

o Record performance metric and completion steps for each student

o Compare and report performance and completion rates between those students whotook academic courses at a technology center and at a comprehensive high school

7. Can the P-20 data system provide, at any point in the pre-K to 20 process, astudents Transcript Plus that includes every school a student attended (includingdates), what courses were taken along with the name of the teacher and the gradethat was received?

o Ensure that the longitudinal database, from pre-K through graduate programs at Oklahoma colleges, includes for each student a complete Transcript Plus that records courses taken, grades received, the name of the teacher for each class

8. Can the P-20 data system accurately track a student who repeatedly drops out,returns, drops out again, etc.?

o Ensure that each student has a student identifier that can track students from pre-K through graduate programs

24 | P a g e


30/32

o Record each time a student drops out o Record if the student subsequently re-enrolls

9. Can the P-20 data system link common school performance to dropping out, to

getting a GED, or to entering college? How many students are on these patterns? o Decide on the measure to be used in the K-12 system to define academic

performanceo Record students who drop out o Compare student performance to drop-outs (see question B.1 for further analysis)o Record GED for students who earn this credentialo Track students who drop out and obtain a GED as they are admitted to collegeo Record for any specified period of time, the number of students who follow this route

10. Can the P-20 data system compare the performance of college students whoentered college directly from high school, entered college after having attendedCareerTech, or entered college after receiving a GED?

o Decide upon the measure of college performance, e.g. remediation, grade point average, persistence, graduation

o Classify incoming college enrollees by their route to college, specifically from highschool, from a GED and with CareerTech experience

o Compare and report college performance by incoming routes, namely, directly fromhigh school, with a GED and with CareerTech experience

11. Can the P-20 data system compare college performance to scores on the ACTamong students who took chemistry or physics with or without a laboratory in highschool?


o Record the ACT score for each student o Record whether the student took chemistry or physics, differentiating between high

school chemistry and physics courses with and without a laboratory o Compare and report college success of students in relation to ACT scoreso Compare and report college success of students with respect to whether students

took high school chemistry or physics with or without a laboratory o Compare and report the relationship between ACT scores and whether students took

high school chemistry or physics with or without a laboratory

12. Can the P-20 data system continuously track the performance of a studentwho takes an AP course, the corresponding AP exam, who then receives collegecredit for the AP course, and then relate each of these steps with the studentsperformance in college in the next level of course work?

o Decide on the measure to be used for judging a college students success ashe or she takes the next level course in a topical areao Record for each student in high school AP courses and AP exam scoreso Record AP credits for each college student o Compare and report college level success in going from one level course to thenext with the students history of AP courses, AP exams and AP credit

13. Can the P-20 data system provide data to compare a students high schoolGPA to the students college GPA (or some other measure of college performance)?

o Record the GPA of each high school student o Record the GPA for each college student o Compare and report the relationship between high school and college GPAo The analysis can be further analyzed in many ways, for example,differentiating this comparison by high school, by college, by area of study, etc.

25 | P a g e


31/32

14. Can the P-20 data system provide data to contrast and consider a highschool student taking honors courses versus AP courses with the students latercollege performance?


o Record the honors courses taken by high school studentso Record the AP courses taken by high school studentso Compare and report the relationship between college success and thestudents experience with high school honors courses and AP courses

15. Can the P-20 data system address the above questions to provide additionaldata so as to also consider ethnicity and/or gender?

o For the above questions, record ethnicity and gender for each pre-K-20student o Conduct the analyses described above, differentiating the results by ethnicity and gender

F. Workforce Preparation

1. What types of jobs are available and obtained by graduates from any specific highschool, CareerTech, and higher education institution? How long do graduatesremain in their initial positions?

o Link high school, CareerTech and college students to OESC employment datao Calculate the number of students from each of the three sources that accept jobs

in Oklahoma businesseso Record the distribution of positions taken by each of the three producing

education/training systems (high schools, CareerTech and college) and record thetime students remain in their initial positions

o This analysis could be made more complex by including field of study and jobclassification, by measures of student performance in each of the three producingeducation/training systems, etc.

2. Can the P-20 data system report whether individual students graduating fromspecific CareerTech programs enter fields in which he or she received training? Canthe P-20 data system relate CareerTech training of an individual student to theearned wages by the student; what are the employment duration and the employersatisfaction?

o Record the fields of training for each student from each CareerTech programo Track each student into the OESC employment datao Record the wage rate of each of these students in his or her initial positiono Record the employment duration and employer satisfaction for each of these

employeeso Compare and record the relationships between CareerTech training programs and

job wages, employment duration and employer satisfaction

Appendix D. BACKGROUND FOR DEFINING THE DATA SYSTEMGOVERNANCEInitially the P-20 Data Coordinating Council will assume the responsibilities of developing the governance for the statewide student longitudinal data system. TheCouncil will evaluate governance mechanisms from other states, with theexpectation that the following policy topics will be addressed:

guidelines for control of data confidentiality and security (privacy protection),access privileges and management

data ownership

26 | P a g e


32/32

access mechanisms for a broad range of users criteria for assessing and auditing data quality, validation and reliability sanctions to ensure responsiveness from agencies in providing data and

analyses mechanisms in the data system to help educators improve instruction

practices and student achievement evaluation methods for ensuring the usability and sustainability of the datasystem

communication mechanisms to encourage the use of the data system guidelines for permissible research using the integrated data system

The Colorado Office of Information Technology has produced a report on acomprehensive approach to state-level data system governance, HB 08-1364Interdepartmental Data Protocol Council Report, that will be useful in addressingthis topic in Oklahoma.

Appendix E. TECHNICAL ADVISORY COMMITTEEAlthough some other states are well along in developing the technical dimensions of statewide student longitudinal data systems, the technical details are complex.

Therefore, the P-20 Data Coordinating Council recommends that it establish a Technical Advisory Committee to recommend the technical details of Oklahomaslongitudinal data system. This Committee will work closely with the P-20 DataCoordinating Council as it establishes the governance structure for Oklahomas datasystem.

Among the technical issues to be addressed are: processes and protocols forcontrolling data confidentiality and security [e.g. adherence to the Family EducationRights and Privacy Act (FERPA) requirements] in the Data Mart and during analyses;processes to implement policies determining data access, data quality, validity andreliability; mechanisms for linkages between and among data systems so data canbe transferred across systems and among interested parties to address questionsthat cut across levels of the educational system and agencies; mechanisms thatenable appropriate access to the unified longitudinal database by a wide range of stakeholders to serve a variety of purposes, including improving of teaching andlearning, informing public policy, fostering a culture of evidence-based decisionmaking and research; evaluation system and program effectiveness and providingreports to various stakeholder groups; and analysis and business management toolsfor the system. In addition, the data system will require data standards, includingdefinition, format, source, provenance, element level and contextual integrity rules;

documentation standards for data elements (data dictionary) and systemscomponents; and data archival and retrieval management systems, along withchange control and change tracking.

p20 coordinating council annual report 2010

Documents