ENGLISH LANGUAGE PROFICIENCY@ UQ

REVIEW OF UQ ENGLISH LANGUAGE POLICY & PROVISION

2010 commissioning of an external review

Campus-wide consultative process: call for submissions, survey and key personnel interviews

Inclusive definition of EL proficiency from the Good Practice Principles for English Language Proficiency for International Students in Australian Universities (AUQA/DEEWR 2009):• “The ability of students to use the English language to make and communicate

meaning in spoken and written contexts whilst completing their university studies.”

Application of GPPs to both international and domestic NESB and ESB students for purposes of the review.

REVIEW OUTCOMES

• Adequacy of current ELP entry requirements• Students not accessing EL support available• Concerns about particular student cohorts• Current support provision

Patterns of areas of concern emerging from submissions:

Reports developed for review suggested no serious ELP performance problems but identified some

cohorts needing greater attention

• ‘refreshment ‘ of ‘the student profile’ to better integrate international and domestic student populations

• re-alignment of ELP definitions and data for planning, programming and monitoring

Analysis of data has facilitated:

GPPs grouped into five themes: 1&2 – University-wide strategy, policy & resourcing; 3&4 – Prospective students and entry standards; 5,6 & 7 – Curriculum design and delivery; 8 & 9 – Transition and social and academic interaction; 10 – Quality assurance

UQ ELP REVIEW AGAINST GPPS

Principle 1: Ensuring students’ ELP is sufficient to participate in studies effectively

Principle 2: Resourcing adequate to meet students’ needs throughout studies

Principle 3: Students responsible for developing ELP and are advised of this early

Principle 4: Entry pathways ensure students can participate effectively

Principle 5: ELP as a graduate attribute

Principle 6: ELP integrated in curriculum

Principle 7: ELP needs to be diagnosed and addressed early with ongoing self- assessment

Principle 8: International students supported from outset to adapt to academic, sociocultural and linguistic environments

Principle 9: International students’ social interaction on/off campus encouraged and supported

Principle 10:Use of evidence to monitor and improve ELP development activities

ELP REVIEW PHASE 2

Phase 2 in 2012 will focus on:• Admission equivalence issues, including for domestic NESB students• Requirements for different disciplines• Post enrolment language assessment (PELA)• Staff development

A university-wide approach to English Language Proficiency

Neil MurraySenior Consultant English Language proficiency

Drivers‘Good Practice Principles for English Language Competence for International

Students in Australian Universities’

The expectation of the project Steering Committee is that universities will consider the Principles as they would consider other guidelines on good practice. As part of AUQA quality audits universities can expect to be asked about the way they have addressed the Principles, just as they are likely to be asked by AUQA auditors about their application of a range of other external reference documents for the university sector (2009: 27).

AUQA ‘s Affirmation (October 2009) AUQA affirms UniSA’s recognition that English language proficiency for

students is a significant and immediate issue that needs to be addressed, and supports timely conclusion of the current discussion about the implementation of the English language proficiency project, including the testing of student proficiency and where required the provision of additional support and guidance for students

http://www.ombudsman.vic.gov.au/resources/documents/Investigation_into_how_universities_deal_with_international_students.pdf

ELP model approved by Academic Board March 2011

A common understanding

• What is English language proficiency?• Shared language• A conceptual framework

Academic proficiency (academic

literacy)

Professional proficiency

(Professional communication skills)

General proficiency

Needed to articulate a framework/approach which is systematic, flexible and sustainable

• Systematic, integrated, equitable,

defensible• Flexible in terms of discipline-specific

needs• Sustainable business case

Framework/Approach

Post-entry English language assessment in

Australian universities: issues and prospects

Catherine Elder,

University of Melbourne

caelder@unimelb.edu.au

AIEC, Adelaide October 20111

OVERVIEW

• Review state of PELA at Australian universities

• Note issues arising from current situation

• Propose future actions

Principle 7Assessment of language development

needs• Students’ English language development needs are diagnosed

early in their studies and addressed, with ongoing opportunities for self‐assessment.

Example of good practice

The university encourages and supports international students (and others) to undertake a diagnostic assessment of their development needs for English language proficiency at a very early stage of their studies. The university offers students opportunities to self‐assess their language skills throughout their studies and to undertake developmental activities in response to the needs they identify.

How prevalent is post-entry language assessment (PELA) in Australian

universities?

• Data gathered as part of a study conducted in 2011 at UoM (Knoch et al. 2011) reveals that

• Of 38 Australian universities surveyed

9 (23%) not using PELA

8 (21%) not using PELA but reviewing policies with a view to introducing PELA

9 (23%) have small Faculty-based PELA initiatives

13 (34%) have university wide policy

• Target population

• Status of test

• Skills assessed

• Mode of delivery

• Reporting of test results

• Interface with language development

How is PELA implemented across different university contexts?

Target population

• All incoming students• International students only• EAL students, whether international or domestic• Particular “at risk” groups (e.g., below 7 on IELTS or equivalent

admission test, below 35/50 on end-of-school English/ESL exam, Foundation students)

• Faculty-specific policies

Issue: negative perceptions of PELA by targeted students

 

 

The thing I have found with DELA is that they are really resistant to do it; particularly local students (native speakers), they feel that it is aimed at international students...even international students will not want to do it. They say well I got in here, why should I sit this test (representative – Commerce Student Centre).

 

[I think that universal testing is better for a whole lot of reasons. Its face validity is better and also there has been so much emphasis on capturing at risk students, this university is supposed to be a university of excellence. How is it identifying which student might want to improve or excel in their English - it’s a sort of remedial model, all this (DELA). So I am very much in favour of universal testing for that reason. (representative of Faculty of Education)

 

Status of assessment

• PELA is optional for all (see GP Principle 3 re student responsibility)• PELA is mandated for all students• PELA Is mandated for particular student categories• Status of PELA differs from faculty to faculty

Issues:

- How to communicate purpose of PELA as aid to language development? (Read, 2008)

- What incentives for participation in cases where PELA is optional?

- How to reconcile notions of compulsion and compliance with notion of PELA as facilitative?

- What sanctions (if any) to impose on non-compliant students where PELA Is mandatory?

Assessment design Skills assessed

• language macro-skills such as reading, listening, writing (speaking less common due to high costs)

• writing only

• language macro-skills plus sub-skills (e.g., vocabulary and grammar)

• language AND numeracy skills

• screening tasks (indirect measures) only or in combination with other skills

Issues: -Different constructs at different institutions – academic language proficiency, academic literacy/ies/ communication skills? (Murray, 2010)

- Limited validity evidence in many cases, including lack of empirical basis for cut-scores (Davies, & Elder, 2004; Elder & von Randow, 2009)

Test delivery and reportingDELIVERY• Pencil and paper• Online (majority) with or without automated scoring• (Repeated) access from home without identity check• Test delivered in laboratory/classroom with identity check

REPORTING• No scores but recommendation regarding support needs sent to student and

Faculty representative• Score with descriptive profile and advice re language development• Writing sample made available if requested• Advisor works through results with student

Issues: - Timing of assessment - how early in the academic year?- How diagnostic and userfriendly are the reports offered to students? (Alderson,

2005, Knoch, forthcoming)

Interface with language development

• PELA results used to direct students to:

- credit bearing courses as part of their degree program

- supplementary short courses targeting particular skills

- discipline-specific tutorials

- one-on-one feedback on essays/assignments

- self-access materials

• PELA results used to stream students into groups

Issues: - How suitable are available development options to diverse population

of students? How tailored to particular assessment profiles?- Should language development courses be mandatory? What sanctions

for non-compliance if so?- How much language development is enough? Who decides?- Are available language development options effective? What evidence?

(…) how do you manage support for the whole cohort of students that will be tested? At the moment the support that is offered varies from faculty to faculty and is quite variable. Some of them direct students to the ASU and coursework subjects and some have own arrangements. Again it’s that follow up – what do the students do with this information? There’s no coordinated approach to the follow up (ASU representative)

 

It seems a lot of the language support stuff looks like it is aimed at international students rather than just being like, short courses - first year students -how to get along in your courses, or something like that. There needs to be different marketing around it (student advisor – Commerce student Centre)

PROSPECTS

• Form cross-university consortium to pool information and resources in relation to PELA

• Establish principles and guidelines for PELA design and delivery

• Set up mechanisms for ongoing evidence-based evaluation of PELA assessment tools and associated language development options

• Consider options and modes for self-assessment in the context of tertiary study

REFERENCES• Alderson, J.C. (2005). Diagnosing foreign language proficiency: the interface between

learning and assessment. London: Continuum.• Davies, A. & Elder, C. (2004) Validity and validation in language testing. E. Hinkel, Editor,

Handbook of research in second language teaching and learning, Lawrence Erlbaum, Mahwah, NJ (2005), pp. 795–813.

• Elder, C., Knoch, U., & Zhang, R. (2009). Diagnosing the support needs of second language writers: Does the time allowance matter? TESOL Quarterly 43.2:351-359.

• Elder, C., & Von Randow, J. (2008). Exploring the utility of a web-based English language screening tool. Language Assessment Quarterly, 5(3), 173-194.

• Knoch, U. (forthcoming) An examination of stakeholder views of diagnostic feedback. Assessing Writing.

• Knoch, U. Elder, C. & McNamara, T. (2011). Report on the feasibility of introducing the Academic English Screening Test (AEST) at the University of Melbourne. Language Testing Research Centre, University of Melbourne

• Murray, N. (2010). Considerations in the post-enrolment assessment of English language proficiency: reflections from the Australian context. Language Assessment Quarterly 7 (4): 343-358.

• Read, J. (2008) Identifying academic needs through diagnostic assessment. Journal of English for Academic Purposes, 3: 180-190.

• .

UQ ENGLISH LANGUAGE PROFICIENCY INITIATIVES IN PRACTICE

Christine Bundesen, Director

Institute of Continuing & TESOL Education, The University of Queensland (ICTE-UQ)

Year Sem 1 Sem 2 Totals2007 95 59 154

2008 98 43 141

2009 137 89 226

2010 218 121 339

2011 164 125 289

Academic Communication Skills (ACS)

English for Academic Communication (EAC)

PATHWAY, INDUCTION & CONCURRENT EL COURSES

Year Sem 1 Sem 2 Totals2007 332 179 511

2008 315 469 784

2009 371 577 948

2010 387 497 884

2011 519 453 972

EAC-PHARMACY SUPPORT COURSE

Revision of materials

Identification and invitation of students

Timetabling of workshops, room bookings

Allocation of English teachers

ICTE

Pharmacy

8 x 2h weekly workshops by ICTE teachers in School of Pharmacy

4 follow-on facilitated practice sessions (support officer)

Analysis by PHRM1012 course coordinator

Semester 1 Semester 2

Enrolment managementICTE-UQ

UQ PHARMACY

EAC-PHARMACY CONCURRENT SUPPORT COURSE

• Co-curricular and skills-based course materials and exercises aligned with assessment tasks

• Key objectives of course:• Identify students whose communication skills are likely to put

them at risk of failure• Provide engaging, discipline-relevant language support • Enhance first year experience of students and facilitate transition

to second year studies• Student reflective data + three direct measures :

• self-evaluated language competency (4.4 to 6.7/10 in 2008, 5.3 to 7.0/10 in 2010)

• number of students predicted to fail semester 2 oral exam consistently reduced from 7-35% to around 1%

• greater commitment/attendance = higher academic achievement across first year assessment

• Indirect measures from focus group data & reflections:• new friendships & peer networking• trust in academic processes• improved confidence• Potentially greater retention & employment opportunities

DEGREE CREDIT-BEARING EL COURSES?

• English language proficiency development through enhanced integration into degree program credit-bearing curriculum – an area which UQ has not to date actioned in any strategic or comprehensive way.

• Should UQ be giving serious consideration both for courses at levels above English language entry requirements and/or for discipline-specific English courses for designated degree programs? If so,• Cumulative and summative assessment for credit-bearing

English courses to be applied in line with policies and procedures for any UQ program.

• Whole-of-university consistency for credit points to be determined across all programs.

• Decision to be made at whole-of-university level as to whether courses would be mandatory or voluntary.

PELA & GRADUATE EXIT TESTING

• In Australia language competence of international students at universities has recently come under intense scrutiny, media attention and public debate.

• Catalysts have been published research reports on EL proficiency of graduates for DIAC skilled migration & DEEWR research showing employers value graduate attributes including language competence.

• Post-enrolment English Assessment (PELA) and graduate exit testing are not yet common practice within Australian universities but are increasing in frequency.

• Good Practice Principles focus on university responsibility to ensure students have English competence to undertake studies, provision of adequate English support throughout student lifecycle, and place responsibility on students to continue language development during studies.

• No PELA currently at UQ but under consideration for Phase 2 of EL Policy Review in 2012.

GRADUATE EXIT TESTING

• First semester 2008 UQ introduced a university-funded voluntary Graduate Exit IELTS test for UG and PG students.

• Mixed views in English language academic community on Exit Testing as positive or negative initiative and on value of mandatory versus voluntary Exit Testing.

• Exit testing is not currently widespread in Australia but anecdotal evidence suggesting this may change as AUQA/TEQSA quality audits demand evidence of improved graduate EL competency.

• Exit Testing from perspective of both institution and student are ways in which a university can specifically show evidence of meeting GPPs #5 and #10.

• Exit Testing can provide students with externalized evidence of language competence to employers or for migration purposes.

• PELA is another source of evidence of how a university can specifically show evidence of meeting GPP #10.

ICTE-UQ GRADUATE EXIT TESTING RESEARCH

• Intended ICTE-UQ research to investigate English proficiency progress from entry to exit through UQ Graduate Exit IELTS test scores.

• Almost 2000 students have taken voluntary UQ Graduate Exit IELTS test between 2008-2011.

• Purposes of research:• consider EL proficiency of test takers compared to

entry thresholds• investigate whether exit scores affected by discipline,

pathway, or nationality• analyse number of students scoring higher, same or

lower than program entry requirement• determine if patterns emerge re most/least

problematic EL macroskill• analyse cross-correlations of data for UG versus PG

students and programs• Research findings in part will inform the way forward for

UQ in tandem with the implementation of the recommendations of the ELP Review.

UQ ENGLISH POLICY & PRACTICE

Thank you.

Christine Bundesen