Assessment
…..Finding a middle way
Quality Assuring Assessment : A Local Authority view
Fife Context
Second/third biggest LA (55,000 pupils) Lean Management Largest schools in Scotland to the smallest Focus on Attainment and Quality Improvement Increase in 5-14 for the last 7 years Increase in SQA in the last 2 years Personalised e-assessment (4 years); PIPS (10
years) A focus on Tracking and Monitoring (E1)
National Context – what is in place
National Debate, AIFL, 4 Capacities National Collection of 5-14 results discontinued Curriculum Framework Skills for learning, life and work Assessment Framework Outcomes and Experiences Commencement of NAR within GLOW BTC 5 and BTC 3 condensed
National Context – what is not yet in place
Success Criteria/standards/milestones of progress Clarity on progression pathways National approach to moderation that will lead to consistent,
valid and reliable judgements Clear Reporting Processes Confidence that our OECD results will improve Clarity on the qualifications performance criteria (Senior
Phase) Clarity on the way success will be measured at a
pupil/school/LA level particularly in the senior phase
Strategy
Energy, Enthusiasm, Enablement, Enrichment
Purpose - agreeing the philosophy
Principles - agreeing the outcomes
Pragmatism - agreeing the actions/timescales
Enablement
Head Teachers leading strategic groups at all levels
Power Point presentations with notes
Reflective questions
An assessment plan outlining actions and timescales and outcomes
Suggested activities with resources for use with school staff and across the cluster
Purpose
Accelerating the learning of individual pupils
Assuring learners and Accountability to interested third parties
Access to further learning opportunities.
Accrediting and Awarding learning against a national and, if possible, international benchmark.
Assessment for Learning
Learners learn best when ... they understand clearly what they are trying to learn, and
what is expected of them they are given feedback about the quality of their work and
what they can do to make it better they are given advice about how to go about making
improvements they are fully involved in deciding what needs to be done
next, and who can give them help if they need it
Assessment as learning
Share learning intentions, success criteria, Learning Goals
Learning through dialogue and self and peer assessment, effective questioning
Metacognition- Learners become aware of how they learn.
Learners participate more in the process of learning.
Assessment of learning
Judgments about pupils’ learning and progress need to be dependable. This means that:
they are VALID (based on sound criteria)
they are RELIABLE (accuracy of assessment and practice)
and they are COMPARABLE (they stand up when compared to judgments in other departments or schools).
Principles Assessment should be valid Assessment should be reliable and consistent Assessment information should be explicit, accessible
and transparent Assessment should be inclusive and equitable Assessment should be an integral part of curriculum
design relating directly to the O’s and E’s Assessment should be manageable Assessment should promote improvements in learning
by providing timely feedback CPD in Assessment should be planned for
Polarised views
“Teacher assessment is unfair because it is unreliable and biased”
“Exams are simply snapshots and are unrepresentative of the work that has really been done”
Trusting teachers’ judgement Harlen 2005
“The findings of the review by no means constitute a ringing endorsement of teachers’ assessment; there was evidence of low reliability and bias in teachers’ judgements”
Assessment Hypothesis
The best teachers will make the best judgments about pupils’ achievements and what next steps they need to do to take to improve further.
Use of assessment information
WE DO WANT: Validity, Reliability Predictability Public and professional confidence in our assessment
system Clarity for learners and their parents in the way
assessment is reported Clear impact on pupils progress
Use of assessment Information
WE DO NOT WANT:
League tables
A blame culture
Increased teacher workload
Constant high stakes testing which limits creativity in the curriculum
Personalised E-assessment
Does NOT replace teachers’ judgement Does NOT replace all the other assessment information Does Not lead to a blame culture Does NOT reduce creativity in the classroom Does significantly reduce teacher work load Improves reliability and validity assessment of
judgements Provides diagnostic information on pupils progress,
class, year, school, LA
Assessment: which matters most?
1. Economy/Society
2. LA
3. School
4. Teacher
5. Parents
6. Pupil
Diagnostic – multi level
Macro diagnostic – what you do with schools and cohorts (E1)
Micro diagnostic – what you do with pupil information (E1)
y = 0.4872x + 44.244
R2 = 0.9612
40
50
60
0 5 10 15 20
SIMD vigintile
PIP
S P
7 A
verag
e
More deprived Less deprived
Macro diagnostic
Q: does this relationship persist across years?
Stage S1
y = 0.4697x + 44.169
R2 = 0.9382
40
50
60
0 5 10 15 20
SIMD vigintile
PIP
S P
7 A
ve
rag
eStage S2
y = 0.504x + 44.044
R2 = 0.9614
40
50
60
0 5 10 15 20
SIMD vigintile
PIP
S P
7 A
vera
ge
Stage S3
y = 0.4674x + 44.387
R2 = 0.8623
40
50
60
0 5 10 15 20
SIMD vigintile
PIP
S P
7 A
ve
rag
e
Stage S4
y = 0.509x + 44.364
R2 = 0.9479
40
50
60
0 5 10 15 20
SIMD vigintile
PIP
S P
7 A
vera
ge
Macro diagnostic
Macro diagnostic
100
150
200
250
40 50 60
PIPS P7 average score
SQ
A S
4 a
verag
e t
arif
f
More deprived Less deprived
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
SIMD vigintile
FM
E (
% o
f ro
ll)
School 1 School 2
Micro diagnostic
10
20
30
40
50
60
70
80
90
0 5 10 15 20
SIMD vigintile
PIP
S P
7 s
co
res
Micro diagnostic
Goodhart’s Law
When a measure becomes a target, it ceases to be a good measure.
Teacher reliability
How should reliability be assessed– By looking at the internal consistency of judgements?
– By looking at the link to external assessments?
– By comparing over time?
– By comparing one teacher with others?
Facets model within Rasch measurement
Personalised E-assessment
Summative test
(Std Grade/
Highers)
Teachers/pupils
Assessment
(with moderation)
Later success – job, degree, salary etc
Assessment: Predictive validity
Finding the Middle Ground
AIFL Communicating clearly with parents, Reporting Defining the standards/success criteria (child
development) Personalised e-assessment Meaningful Moderation (reliable, valid) Senior Phase- assessment systems and
qualifications systems are contiguous Sensitive but serious accountability - analysing
the LA/school/teacher effect
Pragmatism: We need to
1. Actively promote AIFL strategies2. Know what success looks like3. Have effective policy on the use of assessment information4. Reduce teacher workload but improve the reliability and validity of
the assessment system5. Communicate clearly with parents6. Measure what we value and celebrate progress7. Ensure that pace and challenge are not dissipated8. Adapt personalised e-assessments to CFE outcomes9. Be clear on what we mean by moderation10. Ensure that assessment systems and qualification systems
cohere
Pragmatism : KISS principlePragmatism : KISS principle
Application
Higher order thinking
skills
challenge
breadthHOW WELL
HOW MUCH
LEARNERS’ RATE OF PROGRESS
A new language (BTC5)A new language (BTC5)
Pragmatism – Action Plan
Draft reporting template will be disseminated for consultation (June)
Communication strategy for parents is developed (June) Timetable for the phasing out of 5-14 assessments (June) A Fife Success Criteria (the standards) will be developed
for Numeracy and Literacy The Implementation plan for moderation and embedding
AIFL strategies, E- assssment – All our schools for May/June 2011 in
Reading, and Numeracy is established. P1,3,5,7,S2/3 A draft Curriculum and Assessment/ Qualification paper
supporting the Senior Phase and the revised SQA framework to be developed (October)
Appeal: Finding a middle way
Support
Leadership
Accountability for Children’s development