Assessment

…..Finding a middle way

Quality Assuring Assessment : A Local Authority view

Fife Context

Second/third biggest LA (55,000 pupils) Lean Management Largest schools in Scotland to the smallest Focus on Attainment and Quality Improvement Increase in 5-14 for the last 7 years Increase in SQA in the last 2 years Personalised e-assessment (4 years); PIPS (10

years) A focus on Tracking and Monitoring (E1)

National Context – what is in place

National Debate, AIFL, 4 Capacities National Collection of 5-14 results discontinued Curriculum Framework Skills for learning, life and work Assessment Framework Outcomes and Experiences Commencement of NAR within GLOW BTC 5 and BTC 3 condensed

National Context – what is not yet in place

Success Criteria/standards/milestones of progress Clarity on progression pathways National approach to moderation that will lead to consistent,

valid and reliable judgements Clear Reporting Processes Confidence that our OECD results will improve Clarity on the qualifications performance criteria (Senior

Phase) Clarity on the way success will be measured at a

pupil/school/LA level particularly in the senior phase

Strategy

Energy, Enthusiasm, Enablement, Enrichment

Purpose - agreeing the philosophy

Principles - agreeing the outcomes

Pragmatism - agreeing the actions/timescales

Enablement

Head Teachers leading strategic groups at all levels

Power Point presentations with notes

Reflective questions

An assessment plan outlining actions and timescales and outcomes

Suggested activities with resources for use with school staff and across the cluster

Purpose

Accelerating the learning of individual pupils

Assuring learners and Accountability to interested third parties

Access to further learning opportunities.

Accrediting and Awarding learning against a national and, if possible, international benchmark.

Assessment for Learning

Learners learn best when ... they understand clearly what they are trying to learn, and

what is expected of them they are given feedback about the quality of their work and

what they can do to make it better they are given advice about how to go about making

improvements they are fully involved in deciding what needs to be done

next, and who can give them help if they need it

Assessment as learning

Share learning intentions, success criteria, Learning Goals

Learning through dialogue and self and peer assessment, effective questioning

Metacognition- Learners become aware of how they learn.

Learners participate more in the process of learning. 

Assessment of learning

Judgments about pupils’ learning and progress need to be dependable. This means that:

they are VALID (based on sound criteria)

they are RELIABLE (accuracy of assessment and practice) 

and they are COMPARABLE (they stand up when compared to judgments in other departments or schools).      

Principles Assessment should be valid Assessment should be reliable and consistent Assessment information should be explicit, accessible

and transparent Assessment should be inclusive and equitable Assessment should be an integral part of curriculum

design relating directly to the O’s and E’s Assessment should be manageable Assessment should promote improvements in learning

by providing timely feedback CPD in Assessment should be planned for

Polarised views

“Teacher assessment is unfair because it is unreliable and biased”

“Exams are simply snapshots and are unrepresentative of the work that has really been done”

Trusting teachers’ judgement Harlen 2005

“The findings of the review by no means constitute a ringing endorsement of teachers’ assessment; there was evidence of low reliability and bias in teachers’ judgements”

Assessment Hypothesis

The best teachers will make the best judgments about pupils’ achievements and what next steps they need to do to take to improve further.

Use of assessment information

WE DO WANT: Validity, Reliability Predictability Public and professional confidence in our assessment

system Clarity for learners and their parents in the way

assessment is reported Clear impact on pupils progress

Use of assessment Information

WE DO NOT WANT:

League tables

A blame culture

Increased teacher workload

Constant high stakes testing which limits creativity in the curriculum

Personalised E-assessment

Does NOT replace teachers’ judgement Does NOT replace all the other assessment information Does Not lead to a blame culture Does NOT reduce creativity in the classroom Does significantly reduce teacher work load Improves reliability and validity assessment of

judgements Provides diagnostic information on pupils progress,

class, year, school, LA

Assessment: which matters most?

1. Economy/Society

2. LA

3. School

4. Teacher

5. Parents

6. Pupil

Diagnostic – multi level

Macro diagnostic – what you do with schools and cohorts (E1)

Micro diagnostic – what you do with pupil information (E1)

y = 0.4872x + 44.244

R2 = 0.9612

40

50

60

0 5 10 15 20

SIMD vigintile

PIP

S P

7 A

verag

e

More deprived Less deprived

Macro diagnostic

Q: does this relationship persist across years?

Stage S1

y = 0.4697x + 44.169

R2 = 0.9382

40

50

60

0 5 10 15 20

SIMD vigintile

PIP

S P

7 A

ve

rag

eStage S2

y = 0.504x + 44.044

R2 = 0.9614

40

50

60

0 5 10 15 20

SIMD vigintile

PIP

S P

7 A

vera

ge

Stage S3

y = 0.4674x + 44.387

R2 = 0.8623

40

50

60

0 5 10 15 20

SIMD vigintile

PIP

S P

7 A

ve

rag

e

Stage S4

y = 0.509x + 44.364

R2 = 0.9479

40

50

60

0 5 10 15 20

SIMD vigintile

PIP

S P

7 A

vera

ge

Macro diagnostic

Macro diagnostic

100

150

200

250

40 50 60

PIPS P7 average score

SQ

A S

4 a

verag

e t

arif

f

More deprived Less deprived

0

5

10

15

20

25

30

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

SIMD vigintile

FM

E (

% o

f ro

ll)

School 1 School 2

Micro diagnostic

10

20

30

40

50

60

70

80

90

0 5 10 15 20

SIMD vigintile

PIP

S P

7 s

co

res

Micro diagnostic

Goodhart’s Law

When a measure becomes a target, it ceases to be a good measure.

Teacher reliability

How should reliability be assessed– By looking at the internal consistency of judgements?

– By looking at the link to external assessments?

– By comparing over time?

– By comparing one teacher with others?

Facets model within Rasch measurement

Personalised E-assessment

Summative test

(Std Grade/

Highers)

Teachers/pupils

Assessment

(with moderation)

Later success – job, degree, salary etc

Assessment: Predictive validity

Finding the Middle Ground

AIFL Communicating clearly with parents, Reporting Defining the standards/success criteria (child

development) Personalised e-assessment Meaningful Moderation (reliable, valid) Senior Phase- assessment systems and

qualifications systems are contiguous Sensitive but serious accountability - analysing

the LA/school/teacher effect

Pragmatism: We need to

1. Actively promote AIFL strategies2. Know what success looks like3. Have effective policy on the use of assessment information4. Reduce teacher workload but improve the reliability and validity of

the assessment system5. Communicate clearly with parents6. Measure what we value and celebrate progress7. Ensure that pace and challenge are not dissipated8. Adapt personalised e-assessments to CFE outcomes9. Be clear on what we mean by moderation10. Ensure that assessment systems and qualification systems

cohere

Pragmatism : KISS principlePragmatism : KISS principle

Application

Higher order thinking

skills

challenge

breadthHOW WELL

HOW MUCH

LEARNERS’ RATE OF PROGRESS

A new language (BTC5)A new language (BTC5)

Pragmatism – Action Plan

Draft reporting template will be disseminated for consultation (June)

Communication strategy for parents is developed (June) Timetable for the phasing out of 5-14 assessments (June) A Fife Success Criteria (the standards) will be developed

for Numeracy and Literacy The Implementation plan for moderation and embedding

AIFL strategies, E- assssment – All our schools for May/June 2011 in

Reading, and Numeracy is established. P1,3,5,7,S2/3 A draft Curriculum and Assessment/ Qualification paper

supporting the Senior Phase and the revised SQA framework to be developed (October)

Appeal: Finding a middle way

Support

Leadership

Accountability for Children’s development