CONSULTATION ON THE NATIONAL IMPROVEMENT

FRAMEWORK

Carolyn Hutchinson4 December 2015

Improve what?

Learning outcomes for all pupils – focusing on Excellence and Equity

Across the four purposes of CfE: consistent with ‘adaptive competence’ and ‘21st century skills’ conventional bodies of knowledge and strategies for using

and applying this knowledge flexibly personal, interpersonal and social skills, the affective

dimensions of positive beliefs about oneself as a learner skills in monitoring one’s own thinking and learning

Cognitive and emotional dimensions of learning combined and closely connected to motivation

Including the broad, integrated skills of literacy, numeracy and health & wellbeing across learning (BtC5).

Improve how? ‘Excellent learning and teaching is key to raising attainment and

closing the gap in attainment between those in our most and least deprived areas’ (First Minister, Foreword)

Six ‘drivers of improvement’ that will together ensure improved outcomes for learners school improvement school leadership teacher professionalism assessment of children’s progress parental involvement (gathering and using) performance information

Missing are shared understanding of curriculum & progression (what is to be learned) and associated pedagogy (how it can be learned) as well as assessment (how well it has been learned)

Assessment on its own, without close integration with the curriculum or pedagogy, will have limited validity and yield little useful information about pupils' achievement of curriculum outcomes

Evaluating the Improvement Framework:‘Synergies for Better Learning’

Refers to aligning ‘Assessment’ (focused on learning) and ‘Evaluation’ (focused on accountability)

Since the overall purpose of both assessment and evaluation is ultimately to improve student learning: all types of assessment and evaluation practices should

have educational value and practical benefits for those who participate in them, especially students and teachers

Students need to be fully engaged with their learning and empowered to assess/evaluate their own progress (which is also a key skill for lifelong learning and CLPL)

It is important to monitor a wide range of learning outcomes, including the development of critical thinking, social competencies, engagement with learning and overall well-being, which are not amenable to easy measurement (OECD, 2013).

Core Principles of professional learning

Underpinning the Standard for Career-Long Professional Learning are the core principles of professional enquiry... this involves professionals in having an enquiring disposition at the core of their practice

This means they will: think critically and question their own educational beliefs,

assumptions, values and practices create knowledge to enhance, progress and lead the

learning experiences of all learners work collaboratively with colleagues

The teacher as an adaptive expert is open to change and engages with new and emerging ideas about teaching and learning within the ever-evolving curricular and pedagogical contexts in which teaching and learning takes place.

Core Principles practitioner enquiry

Underpinning the Standard for Career-Long Professional Learning are the core principles of practitioner enquiry:

Social Justice … Committing to the principles of democracy and social justice

through fair, transparent, inclusive and sustainable policies and practices

Integrity … Critically examining the connections between personal and

professional attitudes and beliefs, values and practices to effect improvement and bring about transformative change in practice

Trust and Respect … Acting and behaving in ways that develop a culture of trust and

respect

Professional Commitment … Engaging with all aspects of professional practice and working

collegiately with all members of our educational communities with enthusiasm, adaptability and constructive criticality

Performance information: standardised testing

A standardised test: requires all test-takers to answer the same questions, or

a selection of questions from a common bank of questions, in the same way

is scored in a ‘standard’ or consistent way is usually associated with large-scale tests administered

to large samples or populations of pupils, to compare the relative performance of individual pupils or groups of pupils

In Scotland, National Qualifications, the Scottish Survey of Literacy and Numeracy (SSLN) and the OECD’s international PISA survey, are examples of nationally applied standardised assessments

What are the issues around standardised testing?

Debates around standardised testing generally focus on whether they can provide dependable (valid and reliable)

evaluations of student learning the ways in which the data from tests are used (low-stakes or

high-stakes, learning or accountability) whether they have the capacity of themselves to ‘drive’

improvement, and their impact on learners

Newton and Shaw in the UK recently proposed a framework for evaluating assessments that includes: technical quality and fitness for purpose whether the data they provide, along with other relevant

evidence, contribute something new to understanding of pupils’ progress

what would be the human and financial costs and intended and unintended consequences of the assessments for the quality of pupils’ and teachers’ overall experience of learning

Dependability of tests and data There is no such thing as a perfectly valid and reliable test of student

knowledge and skill acquisition. Both tests and data are inevitably subject to some error and bias.

Whether computer-based or not, human judgments affect the technical quality of tests and data: test items, questions and problems that do not align well with published

curriculum outcomes and priorities questions that are ambiguously or poorly-worded, or with cultural bias errors or ambiguities in marking instructions poorly calibrated or misrepresentative performance levels and cut-off scores errors in recording, processing, analysis or reporting of test results difficulties with the choice and use of psychometric models and processes

National data systems are particularly prone to error, given the complexities of collecting data from hundreds of schools on the performance of tens of thousands of pupils. Large-scale education data should therefore be explicitly recognised as estimates.

Tracking over time is especially prone to inaccuracies and errors, making the demand for year-on-year improvement particularly problematic.

Using data from standardised tests

Standardised testing is more likely to be controversial when test scores are used in a ‘high-stakes’ way, to make important public decisions about educational policies, schools, teachers and pupils

It is less likely to be contentious when it is used in a ‘low-stakes’ way to diagnose learning needs and improve learning and teaching in classrooms

Making both high- and low-stakes use of data from the same standardised test is not generally considered to be either straightforward or advisable, since the one is likely to impact on the other in unpredictable ways

Are standardised tests good for learners? Conventional large-scale tests can only evaluate a narrow range of

achievement using limited methods; they will have limited curriculum coverage and diagnostic value for each individual pupil

They may encourage teachers to ‘teach to the test’, focusing on the topics and kinds of tasks that are most likely to be tested, rather than teaching a wider range of knowledge and complex skills

If there is strong pressure on schools and teachers to improve test results, they may reduce the overall quality of teaching and learning for the least confident/borderline pupils, who may be more likely to receive repetitive (rather boring) teaching and test preparation rather than the engaging, challenging and well-rounded programmes offered to their higher-achieving peers

This in turn may reinforce negative stereotypes about their capabilities and limit their future prospects even more

Tests for young children (P1, P4) may be especially prone to error and bias, given their limited attention spans and lack of experience of testing and/or computer-based activities, especially those from less advantaged backgrounds

What are the alternatives?

There are no short cuts to quality in assessment and evaluation Developing and evaluating new tests and the

analysis and use of the data they provide is a detailed, time-consuming and expensive business

The timescale for test development set out in the Framework is quite unrealistic

The alternative may be to start from where we are:Better integrate and use the considerable quantity

of information we already have about our developing system as feedback to inform planning for continuing improvement

Draw on our existing strengths to build an integrated assessment and evaluation framework

Starting from where we are ….

Assessment is for Learning, 2002-2007 to develop teachers’ capacity in assessment to make their own

dependable and consistent judgments about pupils’ learning to explore what a ‘streamlined and coherent system of

assessment’ for Scotland might look like

Circular No. 02, 2005: Assessment and Reporting 3-14 ‘to provide all partners with sufficiently dependable information

and feedback to inform judgments, choices and decisions about learning, and to inform planning for improvement’

All of the various partners in education had an important role to play, providing, sharing and using good quality assessment information to understand and evaluate progress in learning of individuals and

groups of pupils locally in schools from time to time to summarise and report on progress and

achievement in relation to published curriculum ‘standards’, for pupils and their parents and for schools and local authorities across Scotland

Formative

Summative

Internal External

Formative assessmentPersonal learning planning

Involving learners, and parents and other adults, in the learning process

Recognising Achievement

Profiling and reportingTeachers’ judgements, with local moderation as part of quality assurance

Scottish Survey of Literacy and Numeracy in P4, P7, S2

National QualificationsInternational PISA study

School and local authority inspections

Local Authority collection and analysis of information from schools to inform provision and improvement

HMI feedback and subject/ quality/improving reports based on inspections

Aligning assessment and evaluation

…including standardised testingStandardised assessments in the form of the annual Scottish

Survey of Literacy and Numeracy are designed to provide accurate information about overall national standards and over-time trends in achievement in P4, P7 and S2 National sample of pupils, designed to avoid school effects ‘Low stakes’ use of data in order to avoid over-burdening

schools or distorting classroom practice, with individual schools and pupils remaining anonymous

Nationally devised written and practical assessments, closely referenced to CfE, externally marked

Teachers help to draft assessments and act as field officers and external assessors, with some training and external moderation of their judgments

Models approaches to assessment that closely match published curriculum standards (levels), and could provide the basis for a bank of quality assured materials for use in schools for local moderation and building professional capacity in assessment

The survey could also provide….. Clear, interpretable accounts of knowledge and skills

development in and across the various curriculum areas, against which to report on attainment, including over time

In-built cross-stage facility to report on progression across the period P1 to S3 (and evaluate accounts of progression)   

Sampling and reporting at local authority level to provide information about the achievement of pupils that could compare with national data and be used as part of their improvement planning

Information about attitudes, motivation and health & wellbeing through adapted teacher and pupil questionnaires

It would be relatively straightforward to adapt the design and content of the SSLN to current needs and circumstances

Key questions …..

To what extent might the proposed standardised tests in aspects of reading, writing and maths provide information that teachers and schools could use it to improve learning and progress in literacy, numeracy and health & wellbeing across learning?

What would be the likely burden on schools, teachers, pupils? To what extent might the testing preparation and process take up time in classrooms that could be better spent supporting and promoting learning in other ways?

To what extent would scores provide a sufficiently accurate picture of school, teacher and pupil performance to justify their use as a basis for judgments about the quality of schools’ (and LAs’) provision and comparison amongst them?

Is it reasonable to assume that the benefits of standardised testing as proposed in the NIF would outweigh the educational and financial costs (intended and unintended) for pupils, teachers and tax-payers?