GENERAL FRAMEWORK FOR SETTING EXAMINATION

PAPERS: TEST SPECIFICATIONS

PAPER PRESENTED BYW. M. KapambwePrincipal - TEVET

Examinations Council of Zambia

INTRODUCTION

You are a teacher, one day a colleague phones you and she tells you that she is ill. According to the programme she has to develop a test. Due to the illness he/she is not able to develop that test. Of course you are willing to help your colleague.What kind of information do you need to be able to develop the test?

According to The Standards for Educationaland Psychological Testing:

“…the validity of an intended interpretation of test scores relies on all the available evidence relevant to the technical quality of a testing system. This includes evidence of careful test construction…”

Content Definition and MappingWhat content is being measured?How is the measurement of the content domain(s) being operationalised?

Test Specifications– Attainment targets– Assessment objectives (Behaviour or

content)– Taxonomy– Test format– Test plan– Test matrix or test grid

Assessment Options for Different Learning DomainsAll human behavior can be divided into oneof three categories or domains: Cognitive

Affective

Psychomotor

The three categories promote different learning behaviors or objectives. The most commonly assessed behavior domain is the cognitive. Behaviours in the cognitive have been organized into general categories. Such an organization is called taxonomy (system of classification).Owing to the different learning behaviours, different assessment techniques are utilized.

Bloom’s Taxonomy

The Blooms taxonomy is the maintaxonomy used

Test Matrix or GridBehaviours

Know

Comp

Application

Synthesis

Evaluation

Total

1. Knowledge – define, recognize

2. Comprehension – explain, compare

3. Application – Calculate, apply4. Analysis – Analyze, Plan5. Synthesize – Design,

schematize6. Evaluation – Criticize,

Choose, judge

Taxonomy: Bloom

Bloom Taxonomy

Verbs for the cognitive domain Cog. Area

Know. Comp. App. Analysis Synthesis

Evaluation

Behaviours CountDefine

Identify

Label list

Match

Name

Outline

Point out

Quote

Recite

Repeat

Reproduce

Select state

trace

Classify

Compare

Convert

Discuss

Distinguish

Estimate

Explain

Generalise

Give

Examples

Infer

Interpret

Para

Rewrite

Summarize

translate

Change

Compute

Construct

Solve

Demonst-rate

Illustrate

Predict

Breakdown

Differentiate

Investigate

Relate

Separate

Subdivide

Arrange

Combine

Construct

Create

Design

Formulate

Genera-lise

Generate

Group

Integrate

Organise

summarise

Appraise

Conclude

Critique

Critcize

Grade

Judge

Justify

Interpret

Support

Recommend

Old & New Bloom’s Taxonomy

The six levels of Blooms taxonomy were changed from nouns to verbs as shown below.

The New Bloom’s Taxonomy incorporates not only the cognitive domain but also the affective and the psychomotor. This represents a holistic treatment of all the learning domains. The New Blooms Taxonomy has the following domains:– The cognitive - knowledge based domain,

consisting of six levels – The affective - attitudinal based domain,

consisting of five levels, and – The psychomotor - skills based domain,

consisting of six levels.

New Terms Explained

• Remembering: Retrieving, recognizing, and recalling relevant knowledge from long-term memory.

• Understanding: Constructing meaning from oral, written, and graphic messages through interpreting, exemplifying, classifying, summarizing, inferring, comparing, and explaining.

• Applying: Carrying out or using a procedure through executing, or implementing

• Analyzing: Breaking material into constituent parts, determining how the parts relate to each other and to an overall structure or purpose through differentiating, organizing, and attributing.

• Evaluating: Making judgments based on criteria and standards through checking and critiquing.

• Creating: Putting elements together to form a coherent or functional whole; reorganizing elements into a new pattern or structure through generating, planning or producing.

Assessment Options for the Affective Domain

– Difficult to assess due to the personal and internal qualities of affect.

– Inquiry/problem-solving commonly used.– Receiving: observations of learners’discussions

and questionnaires.– Responding: observation of learners’

participation and interviews.– Valuing: Interviews, questionaiires and essays.– Organisation: Observation of learners’choices.– Characterisation: Learners’

responsibilities,projects and debates.

Assessment Options for the Psychomotor

– Hierarchy of difficulty levels ranging from reflex movements to skilled movements.

– Levels generally assessed by observation of either behaviour or performance.

– Observational data can be recorded as an anecdotal record, or by using ckecklists or rating scales.

Romiszowski Taxonomy Knowledge

Facts Concepts

Skills

Reproduction Production

Cognitive (RC) Interactive (Ri)

Psychomotor (Rpm)Reactive (Rr)

Cognitive (PC) Interactive (Pi)Psychomotor

(Ppm)Reactive (Pr)

Romiszowski : Knowledge and Skills (a) Knowledge:

– Factual knowledge: recognizing and remembering of facts and action prescription

– Conceptual knowledge: understanding principles of strategies actions

(b) Skills– Reproductive skills: Routines and vocational

activities based on standard procedures executed according to a clear prescription or protocol.

– Productive skills: Appeals to the creativity and problem solving skills of the student. Application of principles and strategies in new situations.

(c ) Sub-skills for Reproductive and Productive– Cognitive skills: Application of knowledge,

eg. Interpretation, analysis, deciding.– Psychomotor skills: Execution of physical,

motoric actions using knowledge and comprehensive.

– Reactive skills: Attitudes according to a value system.

– Interactive skills: Application of social and communicative aspects in daily life in relation to others, communicating, collaborating.

Romiszowki Taxonomy:

VerbsFactual Knowledge

Conceptual knowledge

Reproductive skills

Productive skill

RecognizeRepeatListNameDefinePoint outUnderline

ExplainInterpretClassifyComparePut in orderCombineDistinguishDescribeGive examples

DemonstrateConstructApply (in concrete situations)

ModelDesignPlanAnalyzeArgueDiscuss

Romiscowski Test Matrix

Subject School Type Year/Class

Author: Period of Testing:

Test Number: Test Format Status: Draft Final

Test Goal/Sub-domain

Content (Summarized/End Term)

Knowledge

Knowledge

Skills Skills

Conclusion

Test construction requires knowledge of the syllabus/course outline on which an examination syllabus is based. There should be congruency or agreement between the learning outcome or objectiveand the testing technique used to assess a learning outcome.

The taxonomy used should be determined by the nature of learning outcome. Not all learning outcomes can be assessed by pencil and paper and within a short time. Most learning outcomes in the affective and psychomotor can be observed by observations and recording over a long time. Performance tasks are also assessed by observation.A mismatch between learning outcomes and assessment procedures leads to what is called curriculum distortion. This distortion could lead to lower learning achievement due to wrong enforcement from the assessment process.

Bibliography

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research AssociationAnderson, L. W. and David R. Krathwohl, D. R., et al (2000) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Allyn & Bacon

Bloom, B.S. and Krathwohl, D. R. (1956) Taxonomy of Educational Objectives: The Classification of Educational Goals, by a committee of college and university Examiners. Handbook I: Cognitive Domain. NY, NY: Longmans, Green. Barnes, J. (1976) 'Introduction' to Aristotle The Nicomachean Ethics ('Ethics'),  Harmondsworth: Penguin.

Barrow, R. (1984) Giving Teaching back to Teachers. A critical introduction to curriculum theory, Brighton: Wheatsheaf Books.MoE, 2003: The Zambia Basic Education Syllabi.Kelly, A.V. (1999) The Curriculum – Theory and Practice, Paul Chapman Publishing Limited, London.

• Cito International(2009) Theory and Practice of Test Construction, Netherlands Fellowship Programme, Arnheim

• Kapambwe, W.M. (2006) The Formative Evaluation of the Continuous Assessment Pilot Programme at Basic School Level in Zambia, Unpublished M.Ed Thesis, University College Dublin

Pickard M.J. New Bloom’s Taxonomy: An Overview for Family and Consumer Sciences. Journal of Family and Consumer Sciences Education, Vol.25, No.1, Spring/Summer 2007.Stenhouse, L. (1975) An Introduction to Curriculum Research and Development, Heinemann Educational Books, London.