Dongkwang Shin (Korea Institute for Curriculum and Evaluation)

Overview of the National English Ability Test

Goals

Directions

Students will acquire basic communicative skills through school education

Level 2: basic academic English Level 3: practical English ▪ Align the contents of NEAT with the National curriculum to reform school education Provide teacher training programs focuses on item development, scoring and instruction with regard to the new test

Students will select either NEAT Level 2 or Level 3 based on their aptitude and future career

Goals and Development Directions

1. Key Characteristics

Test Specifications

Level 2

Level 3

Require the degree of English proficiency necessary for a college or university education

Focus on basic and practical English abilities.

▪ No grammar items ▪ Focus on communicative skills ▪ Topics on Basic Academic English

Characteristics

3000 words 2000 words Vocabulary

English I & II, English reading comprehension & writing, Advanced English conversation

General English, and Practical English conversation

Alignment with

curriculum

▪ No grammar items ▪ Focus on communicative skills ▪ Topics on Practical English: everyday life, work, etc.

Objective

2. Format

Test Domains Number of Items & Test period

Level 2 Level 3 Test period

Listening

Reading

Speaking

Writing

Total

32

32

4

2

70

32

32

4

4

72

40min

50min

15min

35min

140min

In listening and reading domains, anchor items for test equating are included.

Test Specifications

3. Topics : Practical English

Topics Subtopics

Practical

English

1 Everyday life: transportation, telecommunication, shopping, housing, restaurants, hospitals, beauty , etc.

2 Travel & leisure: reservations, transportation, concerts, exhibitions, sports, hobbies, cooking, hotels, public places

3 Family & school life: classes, friends, birthday parties, graduation, counseling, homework, class schedules, examinations, library, test scores

4 Work related: forms, documents, employment, salaries, marketing,

announcements, notices, advertisements, manuals,

Test Specifications

4. Topics : Basic Academic English

Topics subtopics

Basic

Academic

English

1 Humanities, sociology, politics, economy, history, education

2 Science, technology, computer science, information & communication, space, oceans, environment, expeditions

3 Art, literature, anthropology, philosophy

4 Labor, job, career, gender equality, aging society, social welfare,

population, juvenile problems

5 Culture, public morality, public order, civil life, voluntary service,

cooperation

Test Specifications

◦ Inferential and comprehensive understanding items are assessed more in Level 2 than Level 3

◦ The opposite is true for the literal understanding items.

Section Behavioral skill Level 2 Level 3

Listening Literal understanding 60% 68%

Inferential understanding 40% 32%

Reading

Literal understanding 32% 44% Inferential understanding 56% 48%

Comprehensive Understanding 12% 8%

5. Behavioral Domain: Listening & Reading

Test Specifications

Level 2 Level 3

Item Type Number of

Items Item Type Number of Items

Related Interviews 1(4) Answers based on pictures 1(3)

Picture Description 1 Related Interviews 1(4)

Presentation 1 Picture Description 1

Problem Solving 1 Problem Solving 1

Total 4 Total 4

Level 2 Level 3

Item Type Number of Items Item Type Number of Items

Guided Writing about Daily Life 1 Selective Picture Description 1

Complete Detailed Picture description 1

Writing Opinions 1 Writing a Letter 1

Writing based on Picture Cue 1

Total 2 Total 4

- Writing

- Speaking

6. Types: Speaking and Writing

Test Specifications

*A number in a parenthesis indicates the number of sub-questions.

○ Setting up VDI (Virtual Desktop Infrastructure) at the Central Center

○ The students log in on their monitor and take the test at the test center

○ The client in the test center down- loads the test items and uploads the answers in real time

○ Minimize effect of PC environment by providing equivalent test environment

Cloud Computing IBT System

1)Criterion-referenced test A(Excellent), B(Average), C(Basic), and D(Below Basic) Standard Setting (Angoff Method, Bookmark Method)

2) Test equating Students’ scores on different tests are equated Non-equivalent group design with external anchor items

Score Report

Examinees are given a performance level (A, B, C, or D) and its description (performance level description, PLD) for each of the listening, reading, speaking, and writing sections. The NEAT is a criterion-referenced test using test equating.

• The process by which performance cut points (cut scores) are established on an assessment

• As for NEAT standard setting, – twenty panels of faculty, administrators, and

teachers participate in the standard setting – panels expect a minimal competency of each

proficiency level, which is called Angoff Method.

Standard Setting

• Based on test results, students are placed into one of the following four levels: A, B, C, and D, F being the lowest performance level and A being the highest.

• A performance level (A, B, C, or D) and its description (performance level description, PLD) for each of the listening, reading, speaking, and writing.

Standard Setting

Conclusions

NEAT

Enhance students’ speaking and writing skills in addition to listening and reading by adopting NEAT Students can select different tests (Level 2 or Level 3) based on their future career and aptitude Through criterion-referenced test, the students can have clear performance objectives Positive washback effect on school English education

Appendix 1 (Examples of Item Types)

Listening (Advice & Content Agreement)

Appendix 1 (Examples of Item Types)

Reading (Finding Topics)

Appendix 1 (Examples of Item Types)

Speaking (Answers based on the Pictures)

Appendix 1 (Examples of Item Types)

Writing (Writing Opinions)

Thank you!

Overview of the National English Ability Test