2012 Data Coach Training Day 2

Dr. Yuwadee WongbundhitCurriculum and Instruction

Data Coach Day 2 Topics

Proposed FCAT 2.0/EOC Cut Scores

Analyze Student Progress

Create and Export Edusoft Report Builder Files

Add Baseline Data into School Data Base

Data Coach’s Website (Updated)Http://osi.dadeschools.net/datacoach

Designed by

Nelson Suarez

Recap of Day 1Monitoring Tool

for FCAT 2.0 Content Assessed

Benchmarks

Purpose

Features

Overview of FCAT 2.0/EOC Scores

Raw SC, SS and DSS

Achievement Level

Content Scores

Analyzing Baseline

Assessments Report

Average Scores

Average % Correct

% Proficient

Recap of Day 1 (Continued)Data

Analysis

Dig Deeper into Data

Tips for Data Analysis

Protocol and Essential Questions

Data Sources

Acct. Reports

SPI

Acct.-SPI Templates

Excel

Move/Copy Sheet

Freeze, hide, borders,

filter

Read Pivot Table Report

Proposed Cut Scores

6Source: FL DOE/Office of Assessment, October 2011

RawScore

V

RA

LA

IR

Reading Content Scores

2012 FCAT 2.0 ScoresRaw Score

Scale Score(140 to 302 R: G3-10)

(140 to 298 M: G3-8)

Achievement Level

(1 to 5)

V: VocabularyRA: Reading ApplicationLA: Literary AnalysisIR: Informational

Text/Research ProcessSource: FL DOE/Office of Assessment, October 2011

RawScore

C1

C2C3

Algebra Content Scores

2012 Algebra 1 EOC ScoresRaw Score

EOC Scale Score(325 - 475)

Achievement Level

(1 to 5)Source: FL DOE/Office of Assessment, October 2011

FCAT 2.0 Scale Score Ranges

9

Algebra 1 EOC Assessment scale score range: 325 to 475.

Source: FL DOE/Office of Assessment, October 2011

FCAT Achievement Level Descriptions - Policy Definitions

Level 1

Student has little success with the challenging content of the SSS.

Level 2

Student has limited success with the challenging content of the SSS.

Level 3

Student has partial success with the challenging content of the SSS, but performance is inconsistent.

Level 4

Student has success with the challenging content of the SSS.

Level 5

Student has success with the most challenging content of the SSS. A student scoring in Level 5 answers most of the test questions correctly, including the most challenging questions.

Demonstrate an inadequate level of success with the challenging content of the NGSSS.

Demonstrate a below satisfactory level of success with the challenging content of the NGSSS.

Demonstrate a satisfactory level of success with the challenging content of the NGSSS.

Demonstrate an above satisfactory level of success with the challenging content of the NGSSS.

Demonstrate mastery of the most challenging content of the NGSSS.

FCAT FCAT 2.0 and EOC

Grade Level 1 Level 2 Level 3 Level 4 Level 5

3140-181 182-197 198-209 210-226 227-260

4154-191 192-207 208-220 221-237 238-269

5161-199 200-215 216-229 230-245 246-277

6 167-206 207-221 222-236 237-251 252-283

7171-212 213-227 228-242 243-257 258-289

8175-217 218-234 235-248 249-263 264-296

9178-221 222-239 240-252 253-267 268-302

10188-227 228-244 245-255 256-270 271-302

Updated 2012 FCAT 2.0 Reading Cut Scores

Grade Level 1 Level 2 Level 3 Level 4 Level 5

3140-181

(41)182-197

(15)198-209

(11)210-226

(16)227-260

(33)

4154-191

(37)192-207

(15)208-220

(12)221-237

(16)238-269

(31)

5161-199

(38)200-215

(15)216-229

(13)230-245

(15)246-277

(31)

6 167-206(39)

207-221(14)

222-236(14)

237-251(14)

252-283(31)

7171-212

(41)213-227

(14)228-242

(14)243-257

(14)258-289

(31)

8175-217

(42)218-234

(16)235-248

(13)249-263

(14)264-296

(32)

9178-221

(43)222-239

(17)240-252

(12)253-267

(14)268-302

(34)

10188-227

(39)228-244

(16)245-255

(10)256-270

(14)271-302

(31)

Updated 2012 FCAT 2.0 Reading Cut Scores

Updated Reading Cut Scores (140-302)

13Source: FL DOE/Office of Assessment, October 2011

3 4 5 6 7 8 9 10150

175

200

225

250

275

300

182192

200207

213218 222

228

198208

216222

228235

240245

210

221230

237243

249 253 256

227

238246

252258

264 268 271

Reading Scale Score Cuts

Cut 1

Cut 2

Cut 3

Cut 4

Grade

Scal

e Sc

ore

Updated Reading Cut Scores

14Source: FL DOE/Office of Assessment, December 2011

3 4 5 6 7 8 9 10

18% 15% 16% 17% 18% 19% 19% 19%

25% 26% 26% 24% 24% 28% 29% 30%

23% 26% 27% 29% 29%26% 23% 22%

24% 23% 22% 19% 19% 17% 18% 20%

10% 10% 10% 10% 10% 10% 10% 10%

Percentage in each Performance Category

Level 1 Level 2 Level 3 Level 4 Level 5

Grade

Impact: Based Upon 2011 Student Performance in Reading

Grade Source Percent of 2011 Test Takers in Achievement Level

1 2 3 4 5 >=3 3 Reported* 16% 12% 33% 31% 8% 72%

Educators 11% 10% 24% 32% 23% 79%Reactor Panel 18% 25% 23% 19% 15% 57%Rule 18% 25% 23% 24% 10% 57%

4 Reported* 15% 13% 32% 29% 11% 71%Educators 13% 32% 22% 18% 15% 55%Reactor Panel 15% 26% 26% 18% 15% 59%Rule 15% 26% 26% 23% 10% 59%

5 Reported* 16% 15% 33% 28% 9% 69%Educators 14% 30% 28% 14% 14% 56%Reactor Panel 16% 26% 27% 18% 14% 58%Rule 16% 26% 27% 22% 10% 58%

15

*Reported in 2011 on the 2010 FCAT Scale using equipercentile linking

Source: FL DOE/Office of Assessment, October 2011

Impact: Based Upon 2011 Student Performance in Reading

Grade Source Percent of 2011 Test Takers in Achievement Level

1 2 3 4 5 >=3 6 Reported* 17% 17% 32% 26% 9% 67%

Educators 21% 26% 23% 14% 15% 52%Reactor Panel 17% 24% 29% 18% 12% 58%Rule 17% 24% 29% 19% 10% 58%

7 Reported* 14% 17% 33% 24% 11% 68%Educators 12% 15% 40% 24% 10% 74%Reactor Panel 18% 24% 29% 19% 10% 58%Rule 18% 24% 29% 19% 10% 58%

8 Reported* 17% 27% 34% 17% 4% 55%Educators 16% 19% 39% 16% 9% 64%Reactor Panel 19% 26% 29% 15% 11% 55%Rule 19% 26% 28% 17% 10% 55%

16

*Reported in 2011 on the 2010 FCAT Scale using equipercentile linking

Source: FL DOE/Office of Assessment, October 2011

Impact: Based Upon 2011 Student Performance in Reading

Grade Source Percent of 2011 Test Takers in Achievement Level

1 2 3 4 5 >=3 9 Reported* 22% 30% 28% 13% 7% 48%

Educators 26% 19% 35% 9% 12% 55%Reactor Panel 19% 26% 27% 17% 11% 55%Rule 19% 26% 27% 18% 10% 55%

10 Reported* 32% 29% 17% 8% 13% 39%Educators 19% 25% 26% 20% 10% 56%Reactor Panel 19% 25% 26% 20% 10% 56%Rule 19% 25% 26% 20% 10% 56%

17

*Reported in 2011 on the 2010 FCAT Scale using equipercentile linking

Source: FL DOE/Office of Assessment, October 2011

FCAT 2.0 Scale Score Ranges

18

Algebra 1 EOC Assessment scale score range: 325 to 475.

Source: FL DOE/Office of Assessment, October 2011

Grade Level 1 Level 2 Level 3 Level 4 Level 5

3140-182 183-197 198-213 214-228 229-260

4155-196 197-209 210-223 224-239 240-271

5163-204 205-219 220-233 234-246 247-279

6 170-212 213-226 227-239 240-252 253-284

7179-219 220-233 234-247 248-260 261-292

8187-228 229-240 241-255 256-267 268-298

ALG EOC325-374 375-398 399-424 425-436 437-475

GEO EOC

Proposed 2012 FCAT 2.0 Math and Algebra EOC Cut Scores

Grade Level 1 Level 2 Level 3 Level 4 Level 5

3140-182

(42)183-197

(14)198-213

(15)214-228

(14)229-260

(31)

4155-196

(41)197-209

(12)210-223

(13)224-239

(15)240-271

(31)

5163-204

(41)205-219

(14)220-233

(13)234-246

(12)247-279

(32)

6 170-212(42)

213-226(13)

227-239(12)

240-252(12)

253-284(31)

7179-219

(40)220-233

(13)234-247

(13)248-260

(12)261-292

(31)

8187-228

(41)229-240

(11)241-255

(14)256-267

(11)268-298

(30)

ALG EOC325-374

(49)375-398

(23)399-424

(25)425-436

(11)437-475

(38)

GEO EOC

Proposed 2012 FCAT 2.0 Math Cut Scores

Proposed Math Cut Scores (140-298)

21Source: FL DOE/Office of Assessment, October 2011

Proposed Math Cut Scores

22Source: FL DOE/Office of Assessment, October 2011

Impact: Based Upon 2011 Student Performance in Math

Grade Source Percent of 2011 Test Takers in Achievement Level

1 2 3 4 5 >=3 3 Reported* 9% 13% 32% 30% 15% 78%

Educators 16% 23% 36% 16% 9% 62%Reactor Panel 19% 25% 31% 16% 9% 56%Rule 19% 25% 31% 16% 9% 56%

4 Reported* 10% 16% 36% 26% 11% 74%Educators 16% 18% 28% 26% 12% 66%Reactor Panel 19% 23% 28% 18% 12% 58%Rule 19% 23% 28% 20% 10% 58%

5 Reported* 14% 23% 27% 27% 9% 63%Educators 15% 33% 24% 17% 11% 52%Reactor Panel 19% 25% 28% 17% 11% 56%Rule 19% 25% 28% 18% 10% 56%

23

*Reported in 2011 on the 2010 FCAT Scale using equipercentile linking

Source: FL DOE/Office of Assessment, October 2011

Impact: Based Upon 2011 Student Performance in Math

Grade Source Percent of 2011 Test Takers in Achievement Level

1 2 3 4 5 >=3 6 Reported* 23% 20% 29% 19% 9% 57%

Educators 22% 20% 30% 18% 9% 57%Reactor Panel 22% 24% 26% 18% 9% 53%Rule 22% 24% 26% 18% 9% 53%

7 Reported* 19% 19% 32% 21% 9% 62%Educators 20% 24% 28% 16% 12% 56%Reactor Panel 20% 24% 28% 16% 12% 56%Rule 20% 24% 28% 18% 10% 56%

8 Reported* 12% 20% 38% 18% 12% 68%Educators 22% 20% 32% 13% 13% 58%Reactor Panel 22% 22% 30% 14% 12% 56%Rule 22% 22% 30% 16% 10% 56%

*Reported in 2011 on the 2010 FCAT Scale using equipercentile linking

Source: FL DOE/Office of Assessment, October 2011

Proposed Algebra I Cut Scores

25

Source Impact: Percent of 2011 Test Takers in Achievement Level

1 2 3 4 5 >=3 Educators 18% 28% 37% 10% 7% 55%Rule 18% 28% 37% 10% 7% 55%

Source: FL DOE/Office of Assessment, October 2011

Algebra 1 EOC Assessment Scale Score Cuts – Proposed Rule, 10/7/11

Algebra 1 EOC Assessment

Ach. Level 2Cut Score

Ach. Level 3Cut Score

Ach. Level 4Cut Score

Ach. Level 5Cut Score

375 399 425 437

Proposed College-Ready – Achievement Level 3 and Above

Average/Mean Score

Average % Correct

Percent Proficient

What is the difference?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

0

200

400

600

800

1000

1200

1400

1600N

umbe

r of S

tude

nts

Average Score is 17.7 or Average % Correct of 59

67% of Students

36% of Students

P at 70% CutoffProficient at 50% Cutoff

Raw Score Distribution

Digging into Data

Vocabulary (45%)

LA.5.1.6.7(49%)

Base Words

Items: 2, 5, 655%, 20%,

45%

Root

Items: 4, 1070%, 40%

LA.A.5.1.6.3(48%)

LA.5.1.6.8(41%)

Antonyms

Items: 2, 7, 8, 11

Synonyms

Item 2415%

Reporting Category Analysis

Benchmark Analysis

Item Analysis

Identified Students

Content Focus

Disaggregated Data

Grade Level by Content Area

FCAT Level

AYP Subgroup

Grade Level by Content Area

Teacher

Classroom,Period

2011-2012 Baseline Assessment by Subgroup

Analyzing Student Progress

Baseline

Aug. 2011

Fall IA

Oct. 2011

Jan. 2012

2012FCAT2.0

EOC

Apr./May

2012

Winter IA

PG Q1 PG Q2All All

Data Analysis Scheme, Grades 3-5

Grade 3BL, FIA, WIA, FCAT2010-11

Grade 4BL, FIA, WIA, FCAT

Grade 5BL, FIA, WIA, FCAT

Grade 3BL, FIA, WIA, FCAT2011-12

Grade 4BL, FIA, WIA, FCAT

Grade 5BL, FIA, WIA, FCAT

Grade 3BL, FIA, WIA, FCAT2012-13

Grade 4BL, FIA, WIA, FCAT

Grade 5BL, FIA, WIA, FCAT

Data Analysis Scheme, Grades 6-8

Grade 6BL, FIA, WIA, FCAT2010-11

Grade 7BL, FIA, WIA, FCAT

Grade 8BL, FIA, WIA, FCAT

Grade 6BL, FIA, WIA, FCAT2011-12

Grade 7BL, FIA, WIA, FCAT

Grade 8BL, FIA, WIA, FCAT

Grade 6BL, FIA, WIA, FCAT2012-13

Grade 7BL, FIA, WIA, FCAT

Grade 8BL, FIA, WIA, FCAT

Data Analysis Scheme, High School in Reading

Grade 9BL, FIA, WIA, FCAT2010-11

Grade 10BL, FIA, WIA, FCAT

RetakerBL, FIA, WIA, FCAT

Grade 9BL, FIA, WIA, FCAT2011-12

Grade 10BL, FIA, WIA, FCAT

RetakerBL, FIA, WIA, FCAT

Grade 9BL, FIA, WIA, FCAT2012-13

Grade 10BL, FIA, WIA, FCAT

RetakerBL, FIA, WIA, FCAT

Data Analysis Scheme, High School - EOC

Algebra IBL, FIA, WIA, EOC2010-11

Algebra I BL, FIA, WIA, EOC2011-12

GeometryBL, FIA, WIA, EOC

BiologyBL, FIA, WIA, EOC

Algebra I BL, FIA, WIA, EOC2012-13

GeometryBL, FIA, WIA, EOC

BiologyBL, FIA, WIA, EOC

Formulate key questions Obtain data to answer them Find the storyline to bring it all together Determine priority areas for action Share the data with the staff Seek technical assistance if needed Celebrate your achievement results! Use the data to communicate, inform, provoke,

and persuade ... and Improve Your System!

Tips for Analyzing Data

Do you know what your students should know and be able to do each year in

math/reading/science/writing?

Can you identify the gap between where students are right now in relation to high

standards?

What instructional strategies can you use to accelerate students to attain high standards?

Essential Questions

How prepared are you to effectively use assessment to determine ways to alter

instruction to meet high standards?

Do you have effective classroom-level assessments to help you clearly determine

whether students are meeting high standards?

Essential Questions (Cont.)

40

Student Data Sources

Student Data

Sources

Student Performance

Indicators (SPI)

Accountability Reports on

Principal Portal

Edusoft andPMRN

File Download Manager

PMRN – Progress Monitoring Reporting Network

2011-2012 Ideal Data File

41

2011-2012

Data File

2012 Demographics

Information

2011 FCAT Scores

2012 Interim Assessments,

FAIR

2012 FCAT2.0 and EOC Scores

2012 Student Academic

2011 & 2012 Student

Accountability Information

Student Master File

“Student ID” is used as a lookup value to merge the data from different files using Excel formula “V-Lookup”

Master File

Accountability

SPIBaseline

and Interim Assessment

File Download Manager

=VLOOKUP(LOOKUP_VALUE, TABLE_ARRAY, COLUMN INDEX NUMBER, FALSE)

Student ID

Student ID S

tude

nt ID

Student ID

43

Accountability Reports on Principal Portal

Accountability Reports on Principal Portal

Bubble

Reading

Math

Gains

Reading

Math

Regress

Reading

Math

Lowest 25, 35, 45

Reading

Math

45

Accountability Reports on Principal Portal

Accountability Report Data Elements

No. Variable Name

1 Student_ID

2 Name

3 Gain (Y, N)

4 Bubble (P, N, R)

5 Regress (R,N)

6 Sur2_Schl (October FTE)

7 Sur3_Schl (February FTE)

8 Rank (Based on DSS)

9 Most_Recent_FCAT_Grade

10 Most_Recent_FCAT_Level

11 Most_Recent_FCAT_Scale_Score

No. Variable Name

12 Most_Recent_FCAT_DSS_Score

13 Prior_FCAT_Grade

14 Prior_FCAT_Level

15 Prior_FCAT_Scale_Score

16 Prior_FCAT_DSS_Score

17 AYP_Ethnic (A, B, H, I, W, M)

18 LEP_code (LF, LY, LN)

19 LF___2_YR (Y)

20 LEP_Entry_Date

21 SPED_EXCP

22 Lunch_Code (F, R)

Gain(Annual Learning

Gains)

YN

Bubble

PPossibility to be at

the next level

NNeutral

RRisk to decline the

level

RegressRN

Ranking: Percentile Ranking based on DSS

0-25 26-35 36-45 45-100

Principal Accountability ReportSchool Grade View ReportExport Data Print

SPI School Data Elements1. CLASS_SCHOOL

2. STUDENT_NAME

3. STUDENT_ID

4. STUDENT_STATUS_CODE

5. GENDER

6. STUDENT_ETHNICITY

7. STUDENT_GRADE

8. STUDENT_HOMEROOM

9. LEP_Entry_Date

10. LUNCH_CODE

11. Birthdate

12. ESE_INFO

13. LEP_INFO

14. FCAT_CURR_GROUP

15. FCAT_TEST_YEAR

16. FCAT_TEST_MONTH

17. FCAT_GRADE

18. FCAT_NRT_MATH_SCALE

19. FCAT_NRT_MATH_PERCENT

20. FCAT_NRT_MATH_STANINE

21. FCAT_NRT_READ_SCALE

22. FCAT_NRT_READ_PERCENT

23. FCAT_NRT_READ_STANINE

24. FCAT_SSS_MATH_SCORE

25. FCAT_SSS_MATH_LEVEL26. FCAT_SSS_MATH_DEVELOP27. FCAT_SSS_MATH_NS (Rpt. Cat. 1)28. FCAT_SSS_MATH_M (Rpt. Cat. 2)29. FCAT_SSS_MATH_GS (Rpt. Cat. 3)30. FCAT_SSS_MATH_AT (Rpt. Cat. 4)31. FCAT_SSS_MATH_DA32. FCAT_SSS_READ_SCORE33. FCAT_SSS_READ_LEVEL34. FCAT_SSS_READ_DEVELOP35. FCAT_SSS_READ_WP (Rpt. Cat 1)36. FCAT_SSS_READ_MIP (Rpt. Cat 2)37. FCAT_SSS_READ_C (Rpt. Cat 3)38. FCAT_SSS_READ_RR (Rpt. Cat 4)39. FCAT_SSS_SCI_SCORE40. FCAT_SSS_SCI_LEVEL41. FCAT_SSS_SCI_PC 42. FCAT_SSS_SCI_ES 43. FCAT_SSS_SCI_LE 44. FCAT_SSS_SCI_ST45. FCAT_SSS_WRITE_PT46. FCAT_SSS_WRITE_SCORE47. FCAT_READ_PASS_FAIL 48. FCAT_MATH_PASS_FAIL

Student Performance Indicators (SPI)

51

Acct. Files

Reading

Math

SPI Files

SPI data files

Edusoft Files

Baseline

Fall IA

Winter IA

Data Base Files

Reading

Math

Templates

Reading

Math

2011-2012 School Database File Structure

Edusoft Report Builder-1

53

Edusoft Report Builder-2

54

Result

Edusoft Report Builder-3

55

A

B

Result

Edusoft Report Builder-4

56

2. Type Report Name

Note:Create a new folder and name “Edusoft Files”

Save the export file under “Edusoft Files” Folder

Practice Test Day 2Possible Points 45 40 56

Student ID BL RRawSC BL MRawSC BL SRawSC BL R%Corr BL M%Corr

4 40 33 50 5 21 27 40 1 6 10 24 7 42 28 30

15 22 34 21 9 30 17 37

10 15 30 23 20 38 32 17 12 17 25 45 3 35 39 30

57

Go to Data Coach Website and down load “Practice Test Day 2”

VLOOKUP FUNCTION

=VLOOKUP(LOOKUP_VALUE, TABLE_ARRAY, COLUMN INDEX NUMBER, FALSE)

Lookup_Value:What value are you searching for?Excel will look for a match to this value in the leftmost column of the lookup table

Table_Array:Where do you want to search?Use absolute references to “lock” the range by pressing F4 key

Col_index_numWhich column contains the search result?Count over from the first column to figure out what this number should be, starting with 1.

FALSETo force Excel to lookup value be exact match

=VLOOKUP($A3,‘IA File Name'!$B$13:$R$739,3,FALSE)

Column Label

59

Label Column Headers of the Data Base file

BL-Ra

wScBL-%Corr

BL-Perf

BL-CT1Ra

wSc

BL-CT1%Corr

BL-CT1Perf

BL-CT2Ra

wSc

BL-CT2%Corr

BL-CT2Perf

BL-CT3Ra

wSc

BL-CT3%Corr

BL-CT3Perf

BL-CT4Ra

wSc

BL-CT4%Corr

BL-CT4Perf

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Merging File Structure

AJ3 = VLOOKUP($A3,‘IA File Name'!$B$13:$R$739,3,FALSE)

Interim Assessment Data

A1B

2C

3D

4E

5F

17R

Student

Name

Student

ID

Grade

Raw Score

% Correct

Performance

Data Base

A AG or AI

Student ID

Student Name

Accountability Report Data

FCAT

Reporting

Category Scores

Database with Interim Assessment Data

A AI AJ (3) AK (4) AM (5)

Student ID

Student Name

Raw Score

% Correct

Performance

AK3 = VLOOKUP($A3,‘IA File Name'!$B$13:$R$739,4,FALSE)

AM3 = VLOOKUP($A3,‘IA File Name'!$B$13:$R$739,5,FALSE)