ETSGUIDELINES FOR THE ASSESSMENT OF ENGLISH LANGUAGE LEARNERS

ESL 501Article Presentation

Summer Schoenberg

Key Terms

Construct – the skill or proficiency an assessment is intended to measure

Response – any type of performance to be evaluated as part of an assessment

Task – a specific test item, topic, problem, question, prompt, or assignment

Testing accommodation – any change to standardized testing conditions intended to make the test more fair and accessible for an individual or subgroup that does not change the construct being measured

LEP Section 9101 of Title IX states that an LEP

student: Is between the ages of 3 and 21 Is enrolled or preparing to enroll in an

elementary or secondary school Has one of these three profiles:

Not born in the US or speaks a native language other than English

Native American or Alaska native that comes from an environment where a language other than English has had a significant impact on his or her level of English language proficiency

Is migratory, has a native language other than English, and comes from an environment where a language other than English is dominant

Has difficulties in speaking, reading, writing, or understanding the English language that are so severe as to deny the individual one of the following: The ability to meet the state’s proficient level

of achievement on state assessments described in section 1111(b)(3) of the BCLB Act

The ability to successfully achieve in classrooms where the language of instruction is English

The opportunity to participate fully in society

Factors Influencing the Assessment of ELLs

Different Linguistic backgrounds Varying levels of proficiency in English Varying levels of proficiency in native

language Varying degrees of formal schooling in native

language Varying degrees of formal schooling in English Varying degrees of exposure to standardized

testing Varying degrees of acculturation to US

mainstream

Steps within the Planning Process

Test purpose – must be clear in order for valid interpretations to be made on the basis of test scores

Defining the Instruction – definition of what the test is intended to measure

Developing the assessment Specifications – define the content and explain how it will be assessed The state will provide content and performance

standards Tests with more items will supply more reliable scores ELLs should have multiple ways to show what they know Assessments should include a variety of item and

response types

The weight of a task is decided by the importance of the assessed task in relation to other tasks

Assessment specifications will describe how tasks will be presented and how students are then expected to respond

Diagrams, or tables will help ELLs to better demonstrate what they know

Each educational agency should be able to provide information about the cultural backgrounds of its test taking population

Test material should include references to major groups in the tested population and be aware of cultural diversity in readings and illustrations

Developing Test Items and Scoring Criteria

Test items should link content and skill that the item is supposed to measure

All test items should maintain specificity in their match to content guidelines Material should be appropriate and accessible

to examinees and that it only measures the intended construct

Item writers should not assume that students have had any previous experience with given tasks

The criteria for the evaluation should also be made clear to the student

Language should be simplified in directions and given orally or in a language other than English if that will provide for the best understanding of ELLs

Clear and accessible language should be used Colloquialisms and idiomatic expressions should

be avoided Sentence structures should be kept simple The use of negatives should be avoided

Simple context should be used in regards to fictional context

Presentation should clear and consistent ETS Standards for Quality and Fairness should

be applied

External Reviews of Test Materials

Reviews offer an effective technique to improve the quality of assessments

Reviewers are often chosen for their knowledge of the ELL population and specific challenges they may face

Materials must pass committee reviews and should include content experts and professionals who are familiar with issues regarding different ELL populations

Some examples of questions that should be addressed by reviewers:

Does each task match the purpose of the assessment and the assessment specifications?

Are the formats of both the assessment and the response materials appropriate?

Do the tasks and scoring criteria meet standards for fairness?

Evaluating the Tasks Through Tryouts

Useful information is provided through field testing items

Data collected during these tryouts is used to: Inform content and fairness reviews of the

items, Evaluate the clarity of instructions to

examinees, Assess whether ELLs of different proficiency

levels can understand the text of the items

Types of Item Tryouts can include:

One-on-One Interviews

Used to directly obtain information from students about their thought processes while answering the items

Also used to obtain information a bout their understanding of complex language

Small-Scale Pilot Tests

Used to figure out how items will work and how students will respond to items

Large-Scale Pilot Tests

Used to obtain reliable and valid statistics that can be used when selecting items for test forms

Statistics based on these responses are

generally accurate indicators

Limitations of Item Try-outs

Differences in Demographics Curriculum Culture

* all three may make comparisons difficult

Scoring

Rubrics are constructed after determining which English language skills are required for answering a given item

English language skills should play in determining a score

Scorers must have an understanding of the language or presentation style examinees use and will receive training of how to interpret responses and the scoring rubric in a linguistically sensitive way

Testing Accommodations for ELLs

Equity and validity in assessment in the main purpose in providing examinees with testing accommodations

The ETS Guidelines specifically state, “ELLs should have the same opportunity as students who have English as their first language to demonstrate their knowledge or skills in a content area”

Accommodations for ELLscan include changes made to:

Presentation of test materials Students’ responses to test items Scheduling Test setting

Testing modifications are a bit different and can alter what the assessment measures

Accommodations

ELLs currently do not have any type of documentation (like an Individualized Education Plan) that identifies them as being eligible for accommodations

However, if the proficiency level of an ELL is low enough to not produce a valid assessment, the student will receive accommodations

Accommodations

When students receive accommodations in the classroom, the same accommodations are allowed in testing situations, as long as they are appropriate

Students may receive: Direct linguistic support which makes

adjustments to the actual language of a test such as translated texts

Indirect linguistic support which involves changes in the testing environment such as extended time

Statistics are used to evaluate assessment and scoring

Analysis of internal test structure Relations to other variables/constructs Test speededness

These investigations are done by the ETS Standards for Quality and Fairness

Test Scores need to be reliable and valid

Methods of investigating validity include