Scoring methodBachman & Palmer

Two main approaches

One where the score is defined as the number of test tasks successfully completed, so that we count and add up the number of correct responses.

Defining the criteria for what constitutes ‘successful completition’

Determining whether responses will be scored as right or wrong, or with varying degrees of correctness.

Define several levels on one or more rating scales of language ability, and then to rate responses to test tasks in terms of these scales.

Language ability consists of multiple components, and this involves separate, analytic, ratings for each of the specific components included in the contruct definition.

Scale levels should be criterion referenced, with the lowest level on our rating scales defined as ‘no evidence of’ the abililty and the highest level as ‘evidence of mastery of’ the ability.

Specifying the scoring

method

Defining rating scales in terms of the areas of language ability to be assessed.

Determining the number of levels of ability on the different scales.

Rating scales

Componential rating scale

CompensatoryBoth allow profiles of language ability to be reported

Non-Compensatory

Analytic rating scale

Anticipating problems

Demand on resources

Causes of inconsistency

Preparing raters

Obtaining a sufficient number of ratings

Estimating the degree of consistency of ratings

Trialing

The development of a scoring method is

iterative and cycling.

Initial specifications

Try-outAnalysisRevision

Andrea Castillo Bilchi