Application of an On-Line Diagnostic English Language Test

The 11th 334 SymposiumAdvancing Teaching and Learning: Evidence, Outcomes and E-learningLingnan University 4 March 2011

Overview of presentation

Background

The DELTA system

Applications of DELTA

Background

DELTA Context Opportunity provided by planning for 4-year

curriculum for English language centres to review their curricula and provision of learning, teaching and assessment

Tertiary education’s focus on outcomes-based learning, teaching and assessment

Increased emphasis on learner autonomy and the independent learner

Bachman report to UGC on language enhancement

While there may be a number of specific, research and development (R&D) projects involving inter-institutional collaboration among language center staff at different universities, one that is of particular importance, in my view, is the collaborative R&D project involving HKIEd, PolyU, and LU, on the development of a system for assessing English language proficiency ... there is a need for much greater collaboration in this area.

The Bachman Report (1)

Language Enhancement in Hong Kong Universities: Some Observations and Recommendations (Annex A, p.4)Lyle F. Bachman, July 2010

The Bachman Report (2)2. An overall plan for the development of language assessments

could include the following:

Different tests developed for complementary purposes, some could be CR, objectives-based, and some could be NR, and independent of any particular learning objectives.

Tests with some components to be delivered face-to-face and some on-line.

Integration with a language [learning] management system that could include E-portfolios for tracking students’ LE activities and progress in language proficiency.

A delivery platform that is flexible enough to be used for formative and summative course-based assessments by all eight institutions. This flexibility would make it usable for use in general foundational language courses, language for academic purpose courses, and discipline-specific language courses, as well as for certification. (Annex A, p.17)

Relevance and Significance of DELTAThe DELTA project is relevant to the partner institutions and this symposium for several reasons:

It is a genuinely collaborative, inter-university project in which language centres share knowledge, experience and resources.

It provides additional data about students’ language proficiency.

Partner universities will be able to use DELTA results for various purposes.

It will feed into the teaching and learning process with forward perspective to 2012.

It will create the opportunity to engage in collaborative research.

PolyU assessment outline for 4-year curriculum

Entry Exit

What is DELTA?

diagnose students’ strengths and weaknesses track and report on students’ language gains help students to plan their language learning inform curriculum development

An English language proficiency assessment system designed specifically for the Hong Kong academic context which can

Assessment Format Four components - Listening, Reading,

Grammar and Vocabulary

Academic & general text-based

Multiple-choice items

Items calibrated using Item Response Theory (Winsteps) analysis

Assessment targeted to each student’s proficiency level

DELTA Track

DELTA Track and Component Skills Profile

COMPONENT SKILLS PROFILE

Component Diagnostic Report

Grammar

Subskills

phrasal verb gerund part of speech pronoun present perfect Overall proficiency level = DELTA 88 comparative conjunction possessive gerund passive simple present word form infinitive preposition part of speech modal pronoun reflexive pronoun past continuous tense

infinitive The report for Grammar above indicates subskills to work on. Click on the relevant subskill to find resources to help you to improve.

Dif

ficu

lty

The DELTA system

The DELTA SystemENROLMENT /

REGISTRATION

TEST DELIVERY

REPORTING

student-specific test selection

automated scoring & test analysis

Learning Management System (Moodle) and/or e-Portfolio

report generation & transmission

tra

ckin

g

DELTA system (admin)

Customizable set up Test setup (venue, sessions, enrolment

period, etc.) Item database (active / inactive items) Test difficulty level for groups of students Student report content

Management Enrolment and attendance Item database Results (Export to Winsteps for analysis)

New student (1st attempt of DELTA)

Receives email reminder to register

Completes a generic test

Receives an email that results are available

Sets DELTA target (with the advice of a language teacher)

2nd year student (2nd attempt of DELTA)

Receives email reminder to register

Completes a targeted test (targeted to proficiency level)

Receives an email that results are available

(Re)sets DELTA target (with the advice of a language teacher)

Seeks language support to achieve target

Applications of DELTA@ CITYU

Relevance to English Language Teaching and Learning at CityU

Students-Language support (individualised)-Tracking information

Providers of language courses

-Placement (students with no HKDSE/equivalent)

-Formative information

Parent departments & senior management

-Needs report-Tracking information-Overall-Per college/school

DELTA & English support for CityU students from 2012

Year DSE Eng Level 4/+ at entry

DSE Eng Level 3 at entry

4

3 English

2 English English

1 English English

DELTA

DELTA

DELTA

DELTA

Predicted number of test takers

2011approx 1300 (selected Year 1 students)

2012

2013

approx 5200 (all Year 1 students) +

approx 5200 (from 2012) + approx 2600 (all Year 1 students) +

approx 1300 (from 2011)

approx 1300 (from 2011)

http://elws2.cityu.edu.hk/IndependentLearning/writing2.html

http://elws2.cityu.edu.hk/IndependentLearning/writing2.html

IndependeIndependent learning nt learning guidanceguidance

Language Language learning learning

planplan

Meeting Meeting with a with a

Language Language Learning Learning AdvisorAdvisor

Applications of DELTA @Lingnan

Application of DELTA at Lingnan

The DELTA system at Lingnan University is informing and interacting with the teaching-learning process by:

Providing additional information to help in curriculum planning and decision making

Promoting learner autonomy and guiding students in independent learning

Providing valuable input to both students and faculty for academic advising

Curriculum Planning and Decision MakingPlanning and developing the 2012 curriculum

All new courses in 2012 will contain significant Independent Learning (IL) component in which students will plan and execute their individualised learning pathways with help from DELTA student reports

CEAL will track language proficiency development of all students by making the DELTA test mandatory once per year

CEAL currently anticipating proficiency targets and how to work with parent departments

Promoting Learner Autonomy and

Independent Learning DELTA student reports enable students and CEAL staff to identify:specific language subskills for IL workresources that need to be developed

COMPONENT SKILLS PROFILE

Input to provide information and help in Academic Advising

Lingnan currently using Degree Works as a tool for advisors and students in academic advising

Students and staff able to discuss English language proficiency, development and targets

CEAL hopes to work with academic advisors to enable better understanding and more effective use of DELTA student reports

Sample student report in DegreeWorks

The DELTA TeamCentre for Language in Education (HKIEd)

Michelle Raquel, Gwendoline Guan

Centre for English and Additional Languages (LU)

Christine Shirley, Jerry Gray

English Language Centre (PolyU)

Alan Urmston, Carrie Tsang, Felicia Fang, Cherry Ng

English Language Centre (CityU)

Jane Lockwood, Eva Lui

External Consultant

Professor Trevor Bond, James Cook University, Australia