symposium on assessment and learner outcomes · young australians • support all … to become...
TRANSCRIPT
Symposium on Assessment and
Learner Outcomes
Victoria University of Wellington
New Zealand
A Research Focus on Quality
and Equity in Assessment
Val Klenowski Queensland University of Technology
Symposium on Assessment and Learner Outcomes
Victoria University of Wellington
New Zealand
Key Research Projects
• Investigating standards-driven reform in
assessment in the middle years of schooling
• Developing culturally-fair assessment
practices to achieve greater equity and
success for Indigenous students
• Sustainable selves: A new assessment
model for marginalised secondary students
Key Emergent Questions
• What are the implications for teachers‟
assessment practice in the move to a
standards-referenced system?
• How can culture-responsive assessment
and pedagogic practice achieve fairer
assessment?
• How can teachers increase participation for
all students using an electronic portfolio
system to support and assess
achievement?
Current Context in Australia
Background Australian Curriculum
Melbourne Declaration on Education Goals for
Young Australians
• support all … to become successful learners,
confident and creative individuals, and active
and informed citizens,
• promote equity and excellence in education.
• equip all … with the essential skills, knowledge
and capabilities to thrive and compete in a
globalised world and information rich workplaces
of the current century.
• be accessible to all … regardless of their social
or economic background or the school they
attend.
Entitlement Australian Curriculum
• Australian Curriculum
– Learning areas (Phase 1: English, Maths, Science,
History, Phase 2: Languages, Arts, Geography, Phase 3:
Design and Technology, Health and Physical Education,
ICT, Economics, Business and Civics and Citizenship)
– 7 general capabilities
– 3 cross-curriculum priorities.
• In The Shape of the Australian Curriculum: Version 2.0,
… focus on an entitlement for all students. As a result, the
curriculum will articulate what is expected for all students to
learn as well as articulating additional learning options.
(QSA, 2011: 11).
General Capabilities
• Skills, behaviours and dispositions that students develop and
apply to content knowledge and that support them in
becoming successful learners, confident and creative
individuals and active and informed citizens.
• Throughout their schooling students develop and use these
capabilities in their learning across the curriculum, in co-
curricular programs and in their lives outside school.
– Literacy
– Numeracy
– Information and Communication Technology (ICT)
competence
– Critical and creative thinking
– Ethical behaviour
– Personal and social competence
– Intercultural understanding
3 Cross-curriculum priorities
• Aboriginal and Torres Strait Islander histories and cultures will allow
all young Australians the opportunity to gain a deeper understanding and
appreciation of Aboriginal and Torres Strait Islander histories and
cultures, their significance for Australia and the impact these have had,
and continue to have, on our world.
• Asia and Australia’s engagement with Asia will allow all young
Australians to develop a better understanding of the countries and
cultures of the Asia region. Students will develop an appreciation of the
economic, political and cultural interconnections that Australia has with
the region.
• Sustainability will allow all young Australians to develop an appreciation
of the need for more sustainable patterns of living, and to build the
capacities for thinking and acting that are necessary to create a more
sustainable future.
• Embedded in all learning areas as appropriate and have a strong but
varying presence depending on their relevance to the learning area
Assessment
• Teacher assessment
– A to E or “report card”
– Every semester, year and subject
• Parent assessment
– Observation
– After school study
• School based testing
– NAPLAN • Years 3, 5, 7 & 9, Conducted in May
• (Student results in September, diagnostic information Dec/Jan)
– Queensland Comparable Assessment Tasks (QCATs) • Years 4, 6 & 9
– Other diagnostic testing (e.g., PAT-r, PAT-m, DRA)
– Classroom / localised testing
Impact of MySchool
Comparative Analyses and
Competition
Unintended consequences
Cheating
Messages from USA
Shifts in education policy processes
• „Policy creation community‟ in education now includes policy agents and agencies beyond the nation (Mahony, Hextall & Menter, 2004)
• Context of education policy production is changing to now include a complex rescaling across the local, regional, national and global
• This new context for education policy production emphasises a rationale that is expressed in terms of global competitiveness and global imperatives (Rizvi & Lingard, 2010: 15)
Global competitiveness
Global Education Policy Community
(Rizvi & Lingard, 2010: 16)
• Global considerations transform “the balance
between economic efficiency and the social
equity goals of education”.
• OECD policy frame “where market efficiency
concerns now seem to override equity ones”
• Economistic reframing of education,
education policy emphasis on production of
human capital to ensure the competitiveness
of the national economy in the global context
Equity revival?
• Curriculum and assessment policies claim to address equity issues yet transparent accountability, performance pay, and autonomous schools – are more likely to be counter productive to equity goals.
• Unless and until there is a serious attempt to theorise equity as a concept and a practice, the policy rhetoric about equity is unlikely ever to be realised. (Reid, 2011)
• The research outlined in this presentation aims to theorise equity as a concept and practice
My Argument Equity
• In a context of major assessment and curriculum reform there is a need for: – policy and research to support and manage issues of
quality and equity
– up-skilling of teachers – inclusive and ethical practice
– a focus on learning for all students
– teaching and assessment that is responsive to cultural and social difference
• Implications – Care in how the results are interpreted and presented
– See beyond the raw scores - understand the related equity issues
– No over-interpretation of students‟ results in terms of innate ability, dispositions and limitations
Avoiding the Mistakes
• Research informed policy
• Equity theorised as a concept and a
practice
• Opening up participation in learning and
assessment for all students
• Teacher support and development,
particularly in terms of assessment literacy
• Using assessment to support learning
Quality Teachers and Teaching
My Argument Quality
• Increase teacher assessment literacy
• Develop teacher assessment as a source of
dependable results through moderation practice
• Support teacher classroom assessments that directly
contribute to learning
• Develop pre-service teachers‟ assessment capacity
• Initiate and support teacher development of their
theories of learning as the basis for a „principled‟
understanding of learning and assessment
• Encourage use of assessment data for learning
improvement and diagnostic purposes
Assessment and Learning
• Importance of theory and teachers
theorising about their practice (Lawton,1992)
• Sociocultural theory of learning
– Learning a social construction, collective
activities and social transaction
– Learning is both „becoming‟ (increasing in
competency) and „belonging‟ (transformation of
identity) (Murphy, 2009)
– Individual development and social/collective
development are interdependent and
complementary processes
Messages from the research on
standards-based assessment
1. Defined standards serve to:
• inform teacher judgement of system level
expectations
• inform teaching and student learning for
improvement
• support student self- and peer-assessment by
reducing student dependence on the teacher as
the primary or sole source of evaluative feedback.
Principle:
Standards need to be validated through interpretation and
negotiation in moderation practice and should be
empirically derived.
Messages from the research on
standards-based assessment
Messages from the research
2. Moderation:
• Supports teacher judgement
• Achieves fair and dependable judgements
• Responsive to a wide range of evidence types
and assessment contexts
(Klenowski & Wyatt-Smith, 2011)
Implications for policy and practice
Implications
1. Standards-moderation-judgement suite of resources
2. Statement of standards
3. Illustrative Exemplars: folios of work (body of evidence)
or single tasks
4. Processes for arriving at an overall judgement taking
account of trade-offs or compensatory factors:
cognitive commentary
Deep structures of teacher judgement – research in its
infancy.
Moderation is the social practice of exchanging views of
quality for the purpose of comparable and consistent
judgement: inter-rater reliability.
Individual explicit: - that privileged by the
Cartesian model – easily tested
and formalised and held inside
the head.
Group explicit - things expressed explicitly and
used, expressed or transferred
in a group, including stories
about how work is done,
successes or failures, and the
use of metaphors or phrases
that have useful meaning within
a specific group.
Individual tacit: - that associated with skills
- having the “feel” for a skill
- creativity and the creative act
Group tacit: genre; the characteristics of
something whose meanings we
pay little, if any, conscious
attention to; however, our ability
to make sense of what that
„something‟ is, is highly
dependent upon them.
Knowing in action: -The use of knowledge
as a tool of knowing within
situated interaction with the
social and physical world
leading to the production of
new knowledge and
knowing in the
“generative dance”
Fairness in Assessment
• Fairness of any test or assessment depends on whether the students are able to make sense of what is required.
• No cultural neutrality in assessment or in the selection of what is assessed.
• “When setting standards and test content, are we really sure that this is the knowledge we need?
• Are we really privileging certain knowledges to maintain a dominant culture, and, in so doing, ensuring perpetuation of ourselves, as people who have succeeded in the formal educational culture to date?” (Cumming, 2000:4)
Fairness in Assessment
• “No such thing as a fair test and nor could there be as
the situation is too complex and the notion too
simplistic” (Gipps & Murphy, 1994: 273)
• “We will never achieve fair assessment but we can
make it fairer” (Stobart, 2008: 113)
• Assessments have to be as fair as we can make them
- raises issues of access and curriculum as well as
how assessments are framed.
Culture-fair to culture-
responsive assessment
• The opportunity to participate in learning
(access issues) and the opportunity to
demonstrate learning (validity and
fairness in assessment) are deemed
fundamental factors in addressing equity
• Fairness in assessment seen from a
sociocultural perspective not as a technical
concern and is embedded within validity
arguments rather than seen as a separate
concept
Towards Equity
• Equality of access, that is curriculum and
assessment, are equally available to all
groups and are run/presented in such a way
that all groups feel able to participate fully.
(Gipps & Murphy, 1994: 276)
• “Multiple indicators are essential so that
those who are disadvantaged on one
assessment have an opportunity to offer
alternative evidence of their expertise”
(Linn, 1992: 44)
Towards Equity
• “Openness about design, constructs and scoring,
will bring out into the open the values and biases
of the test design process, offer an opportunity for
debate about cultural and social influences, and
open up the relationship between the assessor
and the learner.” (Gipps, 1999: 385)
• Exams and assessments are as fair as possible
with the use of a range of modes and task style
Towards Equity
• Raise teachers‟ awareness of group
differences, interaction of mode of
assessment with construct assessed and
student experience, society‟s views and
expectations of, the abilities of different
student groups, and how this impacts on
expectation, teaching and curriculum offered.
(Gipps & Murphy, 1994: 277)
Fairness and Validity in Assessment
1. “Students from a non-dominant culture experience
testing as a form of cultural intimidation” (Berlack,
2001)
2. These students “may develop attitudes and
practice of resistance to the surveillance,
judgement and categorisation practices that are
affiliated with large-scale testing” (Berlack, 2001)
3. Cultural differences can impact on performance in
the context of standardised tests such as NAPLAN
4. Equity issues relate to opportunity to participate
and to demonstrate learning
Findings Culture-responsive assessment and pedagogy
• Teaching of maths that encompasses the students‟
understandings, dispositions, self-beliefs and
acknowledges their personal view of the value of
learning maths
• Rich tasks, open-ended questioning provide the
basis for authentic problem solving to enhance
personal and intrinsic motivation, perseverance
and resilience
Findings Culture-responsive assessment and pedagogy
• Majority of teachers interviewed had no professional
development in relation to Indigenous cultural
awareness
• Language issues and literacy demands of the
assessments and tests were not always addressed in
pedagogic practice
• Students‟ sociocultural circumstances need to be
understood by teachers and school leaders
• Students‟ attitudes to learning are directly affected by
the value they place on the learning and the success
that they believe they might have in reaching a
satisfactory goal
Effortful Teaching
• Strategic and effortful teaching that encompasses a
diagnostic and holistic view of the student‟s
background, culture, language and demeanour for
developing mathematical thinking
• Capacity of schools to identify “deficit views of
difference” (Ainscow, 2009) students seen as „lacking in
something‟
• Teachers‟ assumptions related to notions of deficit
regarding difference are challenged from a
sociocultural view of learning and assessment – there
is greater respect for valuing of difference
Sustainable Selves Workspace
• Context of the research in Queensland‟s largest re-entry
program, the Flexible Learning Centre Network
• Focus on educational achievements for those who have left
formal education and seeking to re-engage through
alternative programs.
• A new model for assessing the progress of these young
people draws on “authentic assessment” and “assessment
for learning” to develop and implement individual portfolios
• Based on sociological models of capital, these portfolios
compile quantitative and qualitative evidence of young
people‟s resources and achievements.
Context
• The Queensland Government‟s Education
and Training Reforms for the Future (ETRF)
program, to address questions of equity and
opportunity
– Seeks to re-engage students
– To provide for students
– Through socially just or altruistic means
Aims of project
• To enable young people to understand and
document their own education development;
• To provide teachers and para-professionals
with new grounds for curriculum and
counselling;
• To provide systems reporting and tracking of
young people development for funding and
accountability purposes.
Theoretical framework (Connolly, 2011)
A sociocultural approach to assessment for
learning positions teachers as learners in a
highly communicative context.
Communication takes place in the field of
education where agents exchange cultural
capital (Bourdieu‟s reflexive sociology).
Within the field of education, agents exchange
cultural capital through the use of an
assessment for learning approach enacted
largely on an Electronic Portfolio System (EPS).
Electronic Portfolio System (EPS)
• A hybridised electronic portfolio and Content
Management System (CMS); (Social Networking
System) two online systems operating as one.
• Hosted at Queensland University of Technology
• High level of communication diversity
• Provides for unique assessment experiences
(Connolly, 2011)
Access Questions Curricular Questions Assessment Questions
Who gets taught and by
whom?
Whose knowledge is
taught?
What knowledge is
assessed and equated
with achievement?
Are there differences in
the resources available for
different groups?
Why is it taught in a
particular way to this
particular group?
Are the form, content and
mode of assessment
appropriate for different
groups and individuals?
What is incorporated from
the cultures of those
attending?
How do we enable the
histories and cultures of
people of colour, and of
women, to be taught in
responsible and
responsive ways?
Is this range of cultural
knowledge reflected in
definitions of
achievement? How does
cultural knowledge
mediate individuals‟
responses to assessment
in ways which alter the
construct being assessed?
(Stobart, 2005) (Apple, 1989) (Gipps and Murphy, 1994)
References
• Apple, M. W. (1989) „How Equality Has Been
Redefined in the Conservative Restoration‟, in W. G.
Secada (ed.) Equity in Education, New York: Falmer
Press, pp. 7-35.
• Assessment Reform Group. (1999). Assessment for
learning: Beyond the black box. Cambridge,UK:
University of Cambridge School of Education.
• Australian Curriculum Assessment and Reporting
Authority (ACARA). (2011). General capabilities.
Retrieved July 7 2011, from http://www.australiancurriculum.edu.au/GeneralCapabilities
References
• Berlack, H. 2001. Race and the achievement gap. Rethinking Schools Online 15, no. 4. http://www.rethinkingschools.org/archive/_04/Race154.shtml (accessed October 31, 2008).
• Connolly, S. (2011) Towards General Capabilities through Technology: Assessment For Learning as a Field of Exchange, Paper presented at ACACA conference, 4-5 August, Brisbane.
• Cook, S.D.N.& Brown, J.S. (1999) Bridging epistemologies: the generative dance between organizational knowledge and organizational knowing, Organization Science, 10, 4, pp. 381- 400.
• Cumming, J. (2000) „After DIF, What Culture Remains?‟ 26th IAEA Conference, Jerusalem.
• Gipps, C. & Murphy, P. (1994) A Fair Test? Assessment, Achievement and Equity, Buckingham, UK: Open University Press.
• Gipps, C. (1994) Beyond testing: Towards a theory of educational assessment, London: Falmer Press.
• Gutiérrez, K. D., & Rogoff, B. (2003). Cultural ways of learning: Individual traits or repertoires of practice. Educational Researcher, 32(5), 19-25.
References
• Klenowski, V., & Wyatt-Smith, C. M. (2010). Standards, teacher
judgement and moderation in the contexts of national curriculum
and assessment reform. Assessment Matters, 2, 107-131.
• Lawton, D. (1992) Education and politics in the 1990s: conflict or
consensus? London: Falmer Press
• Linn, M. C. (1992). Gender differences in educational achievement.
J. Pfleiderer (Ed.), Sex equity in educational opportunity,
achievement, and testing. [Proceedings of 1991 Educational Testing
Service Invitational Conference] (pp. 11-50). Princeton, NJ:
Educational Testing Service.
• Mahony, P., Hextall, I. & Menter, I. (2004) „Building dams in Jordan,
assessing teachers in England: a case in edu-business‟,
Globalisation, Societies and Education, 2(2): 277-96.
• Murphy, P. (2009) Applying a sociocultural approach to assessment
theory and practice: issues for summative and formative
assessment. The Open University, Presentation at Queensland
University of Technology, Brisbane, 11, September, 2009.
References
• QSA (2011). Time allocations and entitlement: Implementing the Australian curriculum F (P) - 10. Page 11 Retrieved August 3 2011, http://www.qsa.qld.edu.au/downloads/early_middle/ac_time_alloc_entitlement_ report.pdf
• Reid, A. (2011). What sort of equity? Professional Educator, 10(4), 3-4.
• Rizvi, F. & Lingard, B.(2010) Globalizing Education Policy, London: Routledge.
• Rogoff, B. (1995). Observing sociocultural activity on three planes: Participatory appropriation, guided participation, and apprenticeship In J. V.
• Rogoff, B. (2001). Student assessment for the information age. The Chronicle of Higher Education, 48(3), B17.
• Stiggins, R. (2007) Assessment through the student‟s eyes. Educational Leadership, 67, 8, 22-26.
• Stiggins, R. J. (1987). The design and development of performance assessments. Educational Measurement: Issues and Practice, 6, 33-42.
References
• Stobart, G. (2008). Testing times: The uses and abuses of assessment. London: Routledge.
• Stobart, G. (2005) „Fairness in Multicultural Assessment Systems‟, Assessment in Education: Principles, Policy and Practice, 12 (3), 275-87.
• Teese, R., Lamb, S., & Duru-Bellat, M. (Eds.). (2007). International studies in educational inequality, theory and policy (Vol. 1,2,3). Dordrecht: Springer.
• Teese, R. (2006). Condemned to innovate. Griffith Review: Getting smart, the battle for ideas in education, 11(Autumn 2006), 151-159.
References
• Wertsch, J. V. (1991). Voices of the mind: A sociocultural approach to mediated action. London: Harvester Wheatsheaf.
• Wertsch, J. V. (1995). The need for action in sociocultural research. In Wertsch, J.V., P. Del Rio and A. Alvarez (eds) Sociocultural Studies of the Mind. Cambridge: Cambridge University Press.
• Wiggins, G. (1989) A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, 70, 9 (May).
• Wiggins, G. P. (1990). The case for authentic assessment. Practical Assessment, Research & Evaluation, 2(2).