1

Grade Level of Achievement (GLA) Reporting

ASAS Lecture, Universities of Alberta and Calgary

February 6, 2008

2

Introduction

Exploratory Themes The foundation for GLA reporting

Solid classroom assessment theory GLA Reporting: Teacher and Administrator Handbook

Review GLA from two perspectives Can GLA data be collected and reported consistently? Does GLA contribute to our knowledge base regarding

student achievement?

3

Is it possible to balance the Is it possible to balance the decision-making/management decision-making/management information needs of information needs of administrators with teachers’ administrators with teachers’ classroom assessment data classroom assessment data reflecting the learning needs of reflecting the learning needs of students?students?

Key Question…

4

Theoretical Foundation Greater insight is gained when teachers bridge the gap between their assessment efforts and external assessment initiatives Provincial Achievement Tests (PATs)

School Improvement

Grade Level of Achievement (GLA) data

5

A Balanced ModelSee “A Balanced Approach to High Stakes Achievement Testing…” IEJLL Vol. 7, No. 4

FormativeFormative SummativeSummativeN

orm

Ref

eren

ced

Nor

m R

efer

ence

dC

rite

rion

Ref

eren

ced

Cri

teri

on R

efer

ence

dLearner SupportLearner Support

Grade Grade PlacementPlacement DecisionsDecisions

Academic Academic Program Choice Program Choice Advice/DecisionsAdvice/Decisions

Career Choice Career Choice Advice/DecisionsAdvice/Decisions

Program/SchoolProgram/SchoolEffectiveness Effectiveness JudgmentsJudgments

6

“… “… the judgment of the classroom teacher is the judgment of the classroom teacher is an integral part of constructive accountability” an integral part of constructive accountability”

Only when accountability, standards, and Only when accountability, standards, and assessment are fully integrated at the assessment are fully integrated at the classroom level will we achieve the potential classroom level will we achieve the potential for fairness, equity of opportunity, and for fairness, equity of opportunity, and improved academic achievement that teaching improved academic achievement that teaching professionals crave and society demands.professionals crave and society demands.(D. Reeves, (D. Reeves, Accountability For LearningAccountability For Learning, ASCD 2004: 107), ASCD 2004: 107)

7

Theoretical Basis – Understanding by DesignSee Grant Wiggins and Jay McTighe, Understanding by Design, Expanded 2nd edition, ASCD, 2005.

Teaching

Curriculum

Balanced Assessmentcompatible with GLA

Clear Targets

Clear PurposesProp

er M

etho

ds

8

The GLA Reporting Handbook Emphasizes reporting GLA to parents/students as

primary and to Alberta Education as secondary Provides clearer linkages to curricular outcomes Builds on teacher capacity to do good classroom

assessment work Reinforces the link between assessment and

instructional decision-making Emphasizes assessment for learning in ways that

compliment assessment of learning Available on line @

http://www.education.gov.ab.ca/ipr/GLA/TeachAdminHandbook.pdf

Or may be purchased ($6.40 Canadian) @http://www.lrc.education.gov.ab.ca/pro/default.html

9

2006-07 GLA Data Characteristics Alberta Student number is used to align GLA

with related variables No personal identifiers are left in the dataset 3,380 students (1.53%) were on a modified

program (non-graded curriculum) 78% of 217,302 students were non-coded, 2%

had severe disability codes, 8% had mild/moderate disability codes, 1% were coded as gifted and 11% were ESL students

Limitations related to sampling bias

10

For a detailed description of GLA 2005-06 data see… http://education.alberta.ca/media/505076/g

latechnicalreport.pdf 2006-07 data analysis was consistent with

2005-06 data. 2006-07 GLA Pilot Data report to be

published in March.

11

Spearman rho - English LA GLA & Enrolled Grade

Student Groups

2005-06 Correlation Coefficient

(significant at the .01 level -2 tailed)

2006-07 Correlation Coefficient

(significant at the .01 level -2

tailed)

Non Coded .991 .991

Severe Disabilities .853 .868

Mild/Moderate Disabilities

.845 .896

Gifted .994 .994

ESL Canadian Born .974 .966

ESL Foreign Born .925 .945

12

Spearman rho - Math GLA & Enrolled Grade

Student Groups

2005-06 Correlation Coefficient

(significant at the .01 level -2 tailed)

2006-07 Correlation Coefficient

(significant at the .01 level -2 tailed)

Non Coded .994 .992

Severe Disabilities .863 .874

Mild/Moderate Disabilities

.854 .901

Gifted .980 .984

ESL Canadian Born .985 .975

ESL Foreign Born .974 .971

13

Kendall’s tau-b Correlation of PAT by GLA*

2005/06 2006/07

PAT by GLA- Grade and Subject

Corr. Coeff.

PAT by GLA- Grade and Subject

Corr. Coeff.

Gr. 3 Eng. LA .378 Gr. 3 Eng. LA .324

Gr. 6 Eng. LA .406 Gr. 6 Eng. LA .337

Gr. 9 Eng. LA .338 Gr. 9 Eng. LA .323

Gr. 3 Math .388 Gr. 3 Math .342

Gr. 6 Math .403 Gr. 6 Math .366

Gr. 9 Math .399 Gr. 9 Math .409

*all significant at the .01 level -2 tailed

14

Some Conclusions re: Data Utility, Validity and Reliability Correlations between the students’ GLA

and PATs were of moderate strength GLA and PAT correlations were similar in

2006-07 compared to 2005-06. In general, the relationships with enrolled

grade for sub-groups of students were in the expected directions supporting the validity of GLA data

15

Some interesting relationships

Gender Student mobility Student birth month

16

GenderGender - Provincial Mathematics

  Female Male

 Number of students

Percent of total enrolled

(%)Number of students

Percent of total enrolled

(%)

GLA below enrolled grade 9,504 9.0% 11,844 10.6%

GLA equal to enrolled grade 93,918 88.7% 96,508 86.7%

GLA above enrolled grade 464 0.4% 563 0.5%

No GLA provided 2,044 1.9% 2,457 2.2%

Total 105,930 100.0% 111,372 100.0%

Gender - Provincial English Language Arts

  Female Male

 Number of students

Percent of total enrolled

(%)Number of students

Percent of total enrolled

(%)

GLA below enrolled grade 9,335 8.8% 14,781 13.3%

GLA equal to enrolled grade 92,749 87.6% 92,399 83.0%

GLA above enrolled grade 308 0.3% 232 0.2%

No GLA provided 3,538 3.3% 3,960 3.6%

Total 105,930 100.0% 111,372 100.0%

17

Student Mobility and ELA

Mean GLA per enrolled grade as SMI increases in English Language Arts

0

1

2

3

4

5

6

7

8

9

1 2 3 4 5 6 7 8 9 10+

SMI

Gra

de

1

2

3

4

5

6

7

8

9

18

Student Mobility and Math

Mean GLA per enrolled grade as SMI increases in Mathematics

0

1

2

3

4

5

6

7

8

9

1 2 3 4 5 6 7 8 9 10+

SMI

Gra

de

1

2

3

4

5

6

7

8

9

19

Student Mobility and Math

Percent of Students Below Grade Level by Mobility Category for each Grade

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

1 2 3 4 5 6 7 8 9

Enrolled Grade

Perc

ent B

elow

Gra

de

Lev

el

High Mobility Low Mobility

20

English GLA compared to Birth Month, Grade 1

Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 1 students

-2.50-2.00-1.50-1.00-0.500.000.501.001.502.00

1stJan

1stFeb

March April May June July Aug Sept Oct Nov Dec 2ndJan

2ndFeb

z-score Linear (z-score)

21

English GLA compared to Birth Month, Grade 2

Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 2 students

-2.50-2.00-1.50-1.00-0.500.000.501.001.502.00

1stJan

1stFeb

March April May June July Aug Sept Oct Nov Dec 2ndJan

2ndFeb

z-score Linear (z-score)

22

English GLA compared to Birth Month, Grade 3

Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 3 students

-2.50-2.00-1.50-1.00-0.500.000.501.001.502.00

1stJan

1stFeb

March April May June July Aug Sept Oct Nov Dec 2ndJan

2ndFeb

z-score Linear (z-score)

23

English GLA compared to Birth Month, Grade 4

Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 4 students

-2.50-2.00-1.50-1.00-0.500.000.501.001.502.00

1stJan

1stFeb

March April May June July Aug Sept Oct Nov Dec 2ndJan

2ndFeb

z-score Linear (z-score)

24

English GLA compared to Birth Month, Grade 5

Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 5 students

-1.50-1.00-0.500.000.501.001.502.002.50

1stJan

1stFeb

March April May June July Aug Sept Oct Nov Dec 2ndJan

2ndFeb

z-score Linear (z-score)

25

Overall Data Observations There is greater variance in GLA for sub-groups

of students with unique needs GLA data can supplement PAT data with

additional insights for teachers and decision makers

GLA provides missing data GLA value added analysis points to research

needed to better understand factors affecting student achievement

26

13

Related issues from “Effective Student Assessment and Evaluation in the Classroom” “Every model of the teaching/learning process requires

that teachers base their decisions- instructional, grading, and reporting- on some knowledge of student progress toward desired learning outcomes” (Rogers, 1991).

Teachers are responsible for ensuring that the assessment of their students is fair, accurate and reflects the results of a range of assessment methods that consistently, in the teacher’s informed professional judgment, point to the students’ reported assessment result.

27

14

Related Issues, Continued…

Research on teaching reveals that the assessment and evaluation of student learning is unquestionably one of the teacher’s most demanding and complex tasks.

Teacher professional judgment is strengthened through reflective practice and through active participations with teacher colleagues.

28

15

Related Issues, Continued…Purposes of Classroom Assessment and Evaluation

Provide opportunities for students to be actively involved in their own assessments;

Relate to the learning outcomes stated in the programs of study; Focus and motivate students to take the steps necessary to

ensure that learning takes place; and Allow for the identification of students’ strengths and areas of

need.

Student learning is the collective responsibility of all members of the school community. Teachers and experts should work collaboratively to positively support student learning.

Q. How bounded is the “school community in this context?

29

16

Related Issues, Continued… Alberta Education, in consultation with teachers and

administrators, can use the information from provincial and other external testing and assessment programs to identify potential changes in the programs of study and approaches to instruction to improve the learning of

students. When using external testing methods, teacher should

be sure to balance these with information gained from other classroom assessment methods. Fair and reasonable assessment is more accurate when based on the application of a range of assessment

methods. Q. Is GLA a legitimate form of triangulating data for

program planning and evaluation purposes?

30

Lessons Learned Implementation must be a combination of top down and

bottom up efforts Sharing GLA data aligns with emerging transformational

or distributed educational leadership models Students and parents can be more involved in

assessment for and of learning GLA Works best when related to existing assessment

work – Teachers don’t need more work to do Teachers need to be assured that the type of

accountability GLA reporting seeks to serve is learner centered to serve student needs

Teachers benefit from collegial discussions of how to best judge a student’s GLA

31

Lessons Learned Continued

Role of policy entrepreneurs at all levels is vital. GLA R is a project of discovery that will

continue to be refined as it evolves before the full-range of positive impacts result.

Communications is vital. Implementation plan has been responsive to

feedback Trust building is essential.

32

In Conclusion, GLA reporting will be a step towards…

Improved program planning More engaged dialogue among educators about

what works for students A more balanced approach to classroom

assessment Data rich critical reflection within professional

learning communities Better informed leadership focused on

improving the overall quality of education

33

Contacts/additional informationContacts john.burger@gov.ab.ca -

Accountability and Reporting Division, Alberta Education

For more information:

http://www.education.gov.ab.ca/ipr/GLA