national standards for assessing academic english language

Stephen WalkerAcademic Manager - AssessmentUQ CollegeThe University of Queensland

National Standards for Assessing Academic English Language Proficiency on Pathway Programs

This session will begin at 4.55pm (AEST)

Academic English Language Proficiency/Competence

“the ability of students to use the English language to make and communicate meaning in spoken and written contexts while completing their university studies.”

AUQA Good Practice Principles for English language proficiency for international students in Australian universities: final report 2008http://hdl.voced.edu.au/10707/208559

http://hdl.voced.edu.au/10707/208559

Pathway Programs

Courses designed to assist international students with EAL backgrounds in meeting the entry requirements for their chosen university degree courses and to prepare them to commence study in a tertiary EMI context.

What have standards ever done for us?

What are standards?

“Think of standards as an agreed-upon norm that describes the best way of doing something.”ISO - International Organization for Standardization

Why have them?

“Standards are designed to maximize the reliability of the materials, products, methods, and/or services people use every day. ”https://beyondstandards.ieee.org/what-are-standards-why-are-they-important/

https://www.iso.org/home.html

https://beyondstandards.ieee.org/what-are-standards-why-are-they-important/

‘agree upon’ the best way of assessing AELP with ‘pathway students’ in order to ‘maximise the reliability’ of what we do for all our stakeholders.

Academic English language proficiency plays a key role in academic success in EMI contexts, though it’s not the only factor. A minimum, threshold level of competence is required.

(Sawir, Marginson, Forbes-Mewitt, Nyland & Rabia2012; Woodrow, 2006)

We can ‘agree’ that …


• Students usually enter pathway programs using measures received on standardised language tests (IELTS, PTEA, TOEFL, CAE)

• When they commence their pathway program, students usually have not yet met the specified ELP requirements for entry to their university courses

• By the end of their pathway program, students are supposed to have met the specified ELP requirements for entry to their university courses


“There is no accepted assessment framework to determine language progression or learning outcomes in place in the ELT sector in Australia.”

International Education Association of Australia. English Language Competence of International Students Outcomes Report - June 2013 p.9

https://www.ieaa.org.au/documents/item/54

https://www.ieaa.org.au/documents/item/54


“There are no common exit English standards for pathway programs.”

O’Loughlin, K. and Murray, D. (2007), Pathways - Preparation and Selection, a discussion paper preparedfor a National Symposium: English Language Competence of International Students, Sydney, August2007: International Education Association of Australia.

It would be sufficient to have a common language framework (e.g. The CEFR) to describe language competence and that different pathway providers could develop assessments using that framework as the shared standard.

Some have argued that …

However…

“the CEFR alone cannot guarantee that different institutions and stakeholders will use it in a comparable way and come to comparable interpretations when employing and interpreting its proficiency scales.”Claudia HarschHow Suitable Is the CEFR for Setting University Entrance Standards?LANGUAGE ASSESSMENT QUARTERLY 2018, VOL. 15, NO. 1, 102–108

“The only means of ensuring that language levels do have the same meaning across different contexts would be to develop a suite of national tests or a bank of calibrated assessments tasks mapped to the different scale levels. ”

ELICOS Language Levels Feasibility StudyCathie Elder & Kieran O’LoughlinPrepared for the Department of Education Science and Training, 2007

Stephen WalkerAcademic Manager - AssessmentUQ CollegeThe University of Queensland

National Standards for Assessing Academic English Language Proficiency on Pathway Programs

A national standard can be used ...

… as the entire assessment suite used on a pathway program

… as a part of the assessment suite used on a pathway program in conjunction with the centre’s own internally developed assessments

… as a periodic external benchmarking tool to make comparisons between results achieved on internal assessments and the national standard

… not at all - if an institution has other ways of demonstrating how their assessments compare to another means of measuring similar constructs

Institute of Continuing & TESOL Education | CRICOS code 00091C

A national standard will allow us ...

1. … to provide demonstrably fair and valid

assessments supported by rigorous and objective

evidence

Make publically available:

• Constructs – what is being measured?

• Test specifications – how are constructs operationalised?

• Publications – what evidence supports the claims made?

• Research agenda – what areas need additional evidence?


2. … to build trust amongst key

stakeholders

• Students

• Agents

• Teachers

• Faculties/Schools

• Admissions

• Institutions

• Government

Assessment

Literacy

Positive washback

into teaching &

learning



3. … to assess constructs representative of

language use in Australian academic contexts

• Address the specific (academic) language needs of students

planning to study in Australian tertiary institutions (Context validity)

• Include authentic representations of language use in

Australian academic contexts (Cognitive validity)

• Target the proficiency levels specified for entry into specific courses

of study



4. … to leverage economies of scale

• Robust, representative, bias free sampling

• Assessment expertise – test designers, developers, item writers &

psychometricians – These skill sets are not a part of

language teacher education

• Platform development, scalability & hosting

• Validation processes & costs

• External reviewers and researchers



5. … to enhance the international reputation for quality

and rigour of Australian universities and English

language education

• Transparency, clarity & openness

• Evidence based

• Research driven

• External review

• Quality / best practice focus / an art & a science



7. … to meet regulatory compliance (TEQSA / ELICOS

Standards)

https://www.teqsa.gov.au/latest-news/publications/guidance-note-elicos-direct-entry



… to meet regulatory compliance (TEQSA / ELICOS

Standards)

Benchmarking to validated language proficiency frameworks

• Part of the whole of assessment design, development and

validation process. From the ground up.

• A unified approach

• Supported by empirical evidence

• Externally reviewed https://www.teqsa.gov.au/latest-news/publications/guidance-note-elicos-direct-entry



But do we really need

standards here?


Counter arguments

A one size fits all approach to assessment design is not suitable.

All Australian university study contexts share important aspects of academic culture and expectations around academic performance. When selecting constructs for inclusion in the National standard assessments we will focus on those which are shared widely across institutions.

The approach will be inclusive and relevant to all.


Counter arguments

This would take away our unique and competitive advantage

The main factors influencing a student’s decision about where to study are the university, the city and the university course NOT the language centre or the pathway program

Our collective aim is to help students acquire the competencies needed for success in Australian university study. Many of these attributes are shared across all Australian universities so we can easily find enough constructs to target in national standard assessments


Counter arguments

The last thing we need is more regulation telling us how to do things!

Standards are all around us and make our lives fairer and safer and allow us to trust in the systems we use. Imagine if doctors, electricians, or your superannuation company wanted to avoid, or stop following standards and people suffered as a result.

Standards can improve the experience of all our stakeholders.


Counter arguments

In the past when this kind of thing has been suggested it has always been vetoed. The same will happen again!

No, I think it will be seen as an opportunity to secure the long term trust of all stakeholders and to enhance the reputation for quality, of Australian tertiary education, and the Australian English language sector.


How we can begin begin? • Invite participation & consultation amongst centres

• Make use of existing expertise

• Seek agreement on constructs for inclusion

• Identify the evidence needed to develop a

transparent and coherent validity argument

• Ask and answer key questions in

order to produce assessment blueprints

• Invite external review of blueprints


Thank you

Mr Stephen Walker CTEFLA, DTEFLA, MA Applied Linguistics &ELT

Academic Manager – Assessment

UQ College

The University of Queensland Brisbane Qld 4072 Australia

E [email protected] W uqcollege.uq.edu.au

national standards for assessing academic english language

Documents