testing and assessing final presentation.pdf
TRANSCRIPT
-
8/14/2019 Testing and Assessing Final presentation.pdf
1/46
-
8/14/2019 Testing and Assessing Final presentation.pdf
2/46
Objectives
1. To define the basic concepts of languagetesting
2. To specify the reasons for testing3. To identify the basic principles of languagetesting
4. To discuss the different types and forms ofassessment5. To identify the test development procedure
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
3/46
Outline:
Fundamentals of Language Assessment
1. Concept defining :a. Testingb. Assessmentc. Evaluation
d. Measurement2. Purposes for Assessment3. Cornerstones of Assessment: Criteria of a good
Test4. Types of Assessment5. Test Development Procedure
11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
4/46
Evaluation
Assessment
Testing
Figure 1: Relationships among Evaluation, Assessment and Tests
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
5/46
Concept Defining
Evaluation ?
Evaluation is a systematic
analysis of the differentaspects ofan educationalprogram with the aim ofimproving its quality.
Evaluation is theprocess of making
judgments based oncriteria and evidenceand taking decisions .
Evaluation can be defined as thesystematic gathering of information forthe purpose of making decisions (Weiss1972).
Evaluation: The processof determining the worthof something in relation toestablished benchmarksusing assessmentinformation
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
6/46
Concept Defining
Assessment is the process ofgathering and discussinginformation from multiple anddiverse sources in order todevelop a deep understandingof what students know, understand,and can do with their knowledge asa result of their educationalexperiences; the process culminates
when assessment results are used toimprove subsequent learning .
Huba, M. E. & Freed, J. E. (2000). (p. 8)
gathering, recording,
interpreting, using andreporting information about achild s progress and achievementin developing knowledge, skills
and attitudes
Assessment in the Primary School Curriculum:Guidelines for Schools, p. 7, NCCA, 2007
Assessment?
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
7/46
Concept Defining
Assessment?The process of gatheringinformation to monitorprogress and make educationaldecisions .It includes testing but it mayalso include other methodssuch as observation,
interviews, peer assessment,project work , portfolios
Systematic collection andanalysis of information toimprove educationalpractice
Method forunderstanding studentlearning
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
8/46
Concept Defining
TestingA method of measuring apersons ability , knowledgeor performance in a givendomain
(Brown, 2004)
A test is a sample ofbehavior, products,answers, or performancesfrom a particular domain(Carrington, 1994)
it's a systematic method ofeliciting performance which isintended to be the basis forsome sort of decision making "(Hughes, 1989).
Testing is generally
concerned with turningperformance intonumbers.
(Baxter, 1997)Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
9/46
Figure 2 : Relationships among measurement, tests, andevaluation. Bachman (1990)
Concept Defining
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
10/46
Concept Defining
Fundamentals of Language Assessment
Measurement andtest involve thequantification ofobservations.Tests are a type ofmeasurement designedto elicit a specificsample of behavior.
( Bachman 1990)
MeasurementMeasurement : assignmentof numbers (quantity), usesvariety of instrument: test,rating scale. It is theprocess of obtainingnumerical description ofthe degree of individualpossesses. Quantifying ofhow much does a learnerlearned.
11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
11/46
Summary
Evaluation: The placing of value or interpretation of ameasurement. Evaluation encompasses the inferences thatare drawn from measurementsAssessment: A general term used to denote the systematiccollection and interpretation of data that is to be used in
the making of educational decisions, including enhancinginstruction.Test: An instrument or formal process that presents tasksthat yields a specific type of measurement.
Measurement: The explicit quantification of the resultsobtained through testing that is designed to determine thedegree of presence or absence of a particularcharacteristic, quality, or feature.
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
12/46
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
13/46
Purposes for AssessmentAssigning grades or ranks: Assessing students provides the teacherwith at least one type of evidence for grading, graduation, admission.Provide feedback on student learningDetermining one's own instructional effectiveness.Diagnose students strengths and needs
Provide a basis for instructional placementInform and guide instructionCommunicate learning expectationsMotivate and focus students attention and effort
Provide practice applying knowledge and skillsMonitoring students' progressInfluencing public perceptions of educational effectiveness. Students'scores on standardized achievement tests are often used to informthe public about the effectiveness of schools.
Fundamentals of Language Assessment 11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
14/46
Principles of Language Assessment
UsefulnessValidityReliabilityPracticalityWashback EffectAuthenticityTransparency
Security
-
8/14/2019 Testing and Assessing Final presentation.pdf
15/46
Principles of Language Assessment
11/11/2013Fundamentals of Language Assessment
UsefulnessBachman and Palmer s (1996) model of test usefulnessrequires that any language test must be developed with aspecific purpose, a particular group of test takers and a
specific language use in mind.
-
8/14/2019 Testing and Assessing Final presentation.pdf
16/46
Principles of Language Assessment
ValidityThe term validity refers to the extent to which a testmeasures what it says it measures A test is valid if itactually assess the objectives and what has been taught.
Content validity: the test assesses the course content andoutcomes using formats familiar to the studentsConstruct validity: refers to the fit between theunderlying theories and methodology of language learning andthe type of assessmentFace validity: the test looks as though it measures what it issupposed to measure.
-
8/14/2019 Testing and Assessing Final presentation.pdf
17/46
Principles of Language Assessment
ReliabilityIt refers to the consistency of test scores. A reliabletest is consistent and dependable. If you give the sametest to the same students in two different occasions, the
test should yield similar results.Similarly, if we develop two forms of a test that are
intended to be used interchangeably, it should not makeany difference to the test .The student should obtain
about the same score on either form or version of thetest.
-
8/14/2019 Testing and Assessing Final presentation.pdf
18/46
Principles of Language Assessment
11/11/2013Fundamentals of Language Assessment
Three important factors affect test reliability:Test factors : such as the formats , content of thequestions , the length of the exam and the graphicpresentation .
Administrative factors : These include the classroomsetting (lighting, seating arrangements, acoustics, lack ofintrusive noise etc.)Test administration factors : These include affectivefactors like anxiety, motivation, encouragement, etc.
-
8/14/2019 Testing and Assessing Final presentation.pdf
19/46
Principles of Language Assessment
Practicality
An effective test is practical when it:Is not excessively expensiveStays within appropriate time and resourcesconstraintsIs relatively easy to administer
Has a scoring/evaluation/ interpretation procedurethat is specific and time-efficient
-
8/14/2019 Testing and Assessing Final presentation.pdf
20/46
Principles of Language Assessment
Washback
Washback refers to the effects the tests have on teachingand learning.Washback can be positive or negativeCram courses and teaching to the test are examplesof negative washback.
Positive washback can benefit teachers, students andadministrators. In some cases the student may learn whenworking on a test or assessment, teachers reconsidertheir teaching strategies.
-
8/14/2019 Testing and Assessing Final presentation.pdf
21/46
Principles of Language Assessment
AuthenticityAn effective test is authentic when :The language in the test is as natural as possibleItems are contextualized rather than isolatedTopics are relevant and meaningful for learnersSome thematic organization to items is providedTasks represent, or closely approximate, real-world tasks
-
8/14/2019 Testing and Assessing Final presentation.pdf
22/46
Principles of Language Assessment
11/11/2013Fundamentals of Language Assessment
TransparencyTransparency refers to the availability of clear, accurateinformation to students about testing. Such informationshould include outcomes to be evaluated, formats used,
weighting of items and sections, time allowed to completethe test, and grading criteria.
-
8/14/2019 Testing and Assessing Final presentation.pdf
23/46
Principles of Language Assessment
11/11/2013Fundamentals of Language Assessment
SecurityMost teachers feel that security is an issue only in large-scale, high-stakes testing. However, security is part of bothreliability and validity. If a teacher invests time and energy
in developing good tests that accurately reflect the courseoutcomes, then it is desirable to be able to recycle thetests or similar materials.
-
8/14/2019 Testing and Assessing Final presentation.pdf
24/46
Principles of Language Assessment
11/11/2013Fundamentals of Language Assessment
Cornerstones Checklist
When developing, administering and grading exams, ask yourself the following questions:Does your exam test the curriculum content?
Does your exam contain formats familiar to the students?
Does your test reflect your philosophy of teaching?
Would this test yield the same results if you gave it again?
Will the administration of your test be the same for all classes?
Have you helped students reduce test anxiety through test-taking strategies?
Do you have enough time to write, grade and analyze your test?
Do you have all the resources (equipment, paper, storage) you need?
Will this test have a positive effect on teaching and learning?
Are the exam tasks authentic and meaningful?
Do students have accurate information about this test?
Have you taken measures to ensure test security?
Is your test a good learning experience for all involved? Coombe/Hubley
-
8/14/2019 Testing and Assessing Final presentation.pdf
25/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Depending on purpose Depending on characteristics
Screening/Selection/Admission Direct Tests/ Indirect Tests
Placement tests Discrete point/ Integrative tests
Proficiency tests Criterion-referenced/ Norm-referenced tests
Aptitude tests Objective tests/ Subjective tests
Diagnostic tests Speed test/Power testsAchievement tests Knowledge tests/ Skill tests
Progress tests High- stakes /Low-stakes Tests
Formal / Informal tests
Standardized tests / Teacher-made tests
Formative / Summative tests
Assessment for learning / Assessment of learning
Continuous / Terminal Assessment
Alternative Assessment
-
8/14/2019 Testing and Assessing Final presentation.pdf
26/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Depending on purpose :
Screening/Selection/Admission : To know if a personhas the required behavior to be successful in a specificprogram (not based on objectives), e.g. IPCs admission test.
Placemen t: To determine the level in which a personshould be placed inside a language program. Typically, theyassign students to classes at different levels.
Proficiency : To measure a person s overall ability orproficiency in a language. e.g The TOEFL test.
-
8/14/2019 Testing and Assessing Final presentation.pdf
27/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Aptitude : To know the talents of a person to do somethingspecific and measure the suitability of a candidate for a specificprogram of instruction. It predicts a person s success prior toexposure to a language course.
Diagnostic : To identify learners' strengths and weaknesses. Ithelps teachers to make decisions on what needs to be taught
Achievement : To know if a determined objective has beencovered successfully. It measures what learners have learnt ona language course - usually given at the end of the course
Progress : To check improvement achieved according to areferential point in a program. It measures a learners' progressduring a language course
-
8/14/2019 Testing and Assessing Final presentation.pdf
28/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Depending on characteristics:
Direct Tests : Test the skill itself. Students performexactly what we want to test.
Indirect Tests : Test abilities related to the skills we areinterested in, e.g. assess grammar and spelling through awritten exercise.
Discrete point : Focus on one linguistic element at atime. e.g. cloze exercise on verb tenses .
Integrative tests : Measure overall language proficiency,e.g. oral interviews = fluency, pronunciation, content, grammar,comprehension, etc.
-
8/14/2019 Testing and Assessing Final presentation.pdf
29/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Norm-Reference Criterion-ReferenceDefinition Norm-Referenced tests measure the
performance of one group of test takersagainst another group of test takers.
Criterion-Reference tests measure theperformance of test takers against thecriteria covered in the curriculum.
Purpose To measure how much a test taker knowscompared to another student.(to classify)
To measure how much the test taker knownbefore and after the instruction is finished.
Content Norm-Referenced tests measure broadskill areas taken from a variety oftextbooks and syllabi.
Criterion-Reference tests measure the skillsthe test taker has acquired on finishing acurriculum.
Administration Norm-Referenced tests must beadministered in a standardized format.
Criterion-Reference tests need not beadministered in a standardized format.
Scorereporting
Norm-Referenced test scores arereported in a percentile rank.
Criterion-Reference test scores are reportedin categories or percentage.
Scoreinterpretation
In Norm-Referenced tests, if a test takerranks 95%, it implies that he/she hasperformed better than 95% of the othertest takers.
In Criterion-Reference, the score determineshow much of the curriculum is understoodby the test taker.
-
8/14/2019 Testing and Assessing Final presentation.pdf
30/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Objective tests : No judgment involved. Answers are eitherright or wrong. (e.g. Yes/no question items)
Subjective tests : Judgment and opinion involved. No right orwrong answer. (e.g. opinion/discussion items)
-
8/14/2019 Testing and Assessing Final presentation.pdf
31/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Speed test : Easy items in a very short time. Assessspeed of performance and strategy, e.g. scanning exercises.
Power test : Difficult items in enough time. Assess
knowledge.
Knowledge tests : Assess the language components,
e.g. grammar quizzes. Skill tests :Assess the skills, e.g. listening quizzes.
-
8/14/2019 Testing and Assessing Final presentation.pdf
32/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
High-stakes tests are those where the results are likely to have amajor impact on the lives of large numbers of individuals, or onlarge programs. E.g. Bac, TOEFLLow-stakes tests are those where the results have a relativelyminor impact on the lives of the individual or on small programs.(in-class progress tests or quizzes).
Informal assessment : It is the observation of everyday performance.It is a way of collecting information about students performance innormal classroom conditions without establishing test conditions. It cantake a number of forms like unplanned comments, verbal feedback,observing students perform a task or work in small groups, etc.Formal assessment : uses formal tests or structured continuousassessment to evaluate a learner's level of language. It is systematicallyadministered to gauge the proficiency level of the person taking the test.It takes a number of forms( formative, summative, achievement, etc.)
-
8/14/2019 Testing and Assessing Final presentation.pdf
33/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Teacher -made tests: oral and written tests made up byan individual teacher for his/her class. They are notcommercially standardized and produced. These tests canbe quite difficult and time consuming.
Standardized tests: contain standardized andpredetermined procedures for its administration, scoringand for the interpretation of results. They are designed tobe used with large populations of students, compare
students performance and are reported in quantitativeterms.
-
8/14/2019 Testing and Assessing Final presentation.pdf
34/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Formative Assessment: is the process of gathering ongoinginformation during instruction to determine what studentsknow and can do, and to provide descriptive feedback toimprove learning and inform teaching. It aims to evaluate theeffectiveness of learning at a time during the course , inorder to provide feedback to make future learning moreeffective. It is carried out frequently and is planned at thesame time as teaching.
Summative Assessment: refers to forms of assessmentused to evaluate student learning at the end of an instructional
unit by comparing it against some standard or benchmark. It isthe evaluation, or judgment reached at the end of a topic,theme, unit, semester, term, or school year based onperformances/ products ,midterm exams, final projects, etc.
-
8/14/2019 Testing and Assessing Final presentation.pdf
35/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Assessment of learning Assessment for LearningWhen ? happens after the learning takes
placeAn intergral part of the learningprocess
Who? Information is gathered by theteacher
Information is shared with thelearner
What? Information is transformed intomarks or grades
Information is available on thequality of learning
Why? Check status- Comprison withthe performance of others
Improve learning- Comparisonwith aims and objectives isimportant
Purpose Looks back on past learningpurpose
Looks forward to the next stage oflearning
-
8/14/2019 Testing and Assessing Final presentation.pdf
36/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Terminal assessment : occurs at the end of a course ormajor unit (examination, practical or situational assessment,dissertation, thesis or portfolio). It is carried out forsummative purposes (progress to the next stage of the course,accreditation after having satisfactorily completed a module,
unit, etc., or to contribute to their final mark, grade or degreeclassification).Continuous assessment . It is carried out on an on-goingbasis while students are actually working through a course. It
can take a wide range of forms , including periodic tests, quizzesessays and other types of assignment, on-going assessment ofpractical work. It can take the form of a mixture of formativeand summative assessment.
-
8/14/2019 Testing and Assessing Final presentation.pdf
37/46
Types of Assessment
11/11/2013Fundamentals of Language Assessment
Alternative assessmentAlternative assessment is any assessmentmethod that is an alternative to traditionalpaper-and-pencil tests . Alternativeassessment requires students todemonstrate the skills and knowledge thatcannot be assessed using a timed shortanswer test. It focuses on authenti c andinteractiv e tasks. AA seeks to revealstudents' critical-thinking and evaluationskills by asking students to complete open-ended tasks that often take more than oneclass period to complete.
Source:http://teachingtoday.glencoe.com/howtoarticles/alternative- assessment-primer
AuthenticPerformance-basedInteractiveFormativeFocus on high-order skillsIntegrativeExpansive
-
8/14/2019 Testing and Assessing Final presentation.pdf
38/46
ReportsRole plays Brochures DiscussionsInterviewsAudio and VideoRecordings
Presentations
Artifacts
Anecdotal
Records
PostersBanners
Checklists, RatingScales, and
Rubrics
Portfolios Designing mindmaps JournalsDiariesDisplays
Conductingsurveys
Designingcartoons and
commentsConferences
Writing andacting outscenarios
Fundamentals of Language Assessment
Some Alternative Assessment activities
11/11/2013
-
8/14/2019 Testing and Assessing Final presentation.pdf
39/46
Test Development Procedure
11/11/2013Fundamentals of Language Assessment
Planning
Test Content and Development
Before the Test
Test Administration
After the Test
Reflection on the testing process
-
8/14/2019 Testing and Assessing Final presentation.pdf
40/46
Test Development Procedure
11/11/2013Fundamentals of Language Assessment
PlanningEstablish purpose of testo place students in programo achievement of course outcomeso diagnosis of strengths and areas for improvement
Identify objectivesMake an inventory of course content and materials
o consider appropriate formatso establish overall weighting
Write test specifications
-
8/14/2019 Testing and Assessing Final presentation.pdf
41/46
Test Development Procedure
11/11/2013Fundamentals of Language Assessment
Test Content and DevelopmentMap the examo decide on sections, formats, weighting
Construct items according to test specificationsEstablish grading criteriaPilot the exam
-
8/14/2019 Testing and Assessing Final presentation.pdf
42/46
Test Development Procedure
11/11/2013Fundamentals of Language Assessment
Before the TestProvide information to studentso coverage, weighting, formats, logistics
Prepare studentso student test-taking strategieso practice exam activities
-
8/14/2019 Testing and Assessing Final presentation.pdf
43/46
Test Development Procedure
11/11/2013Fundamentals of Language Assessment
Test AdministrationDecide on test conditions and proceduresOrganize equipment neededEstablish test-taking policy
Inform students about availability of results
-
8/14/2019 Testing and Assessing Final presentation.pdf
44/46
Test Development Procedure
11/11/2013Fundamentals of Language Assessment
After the Test
Grade testsCompute basic statisticsGet results to students and provide feedbackConduct exam analysisReport on exam results
-
8/14/2019 Testing and Assessing Final presentation.pdf
45/46
Test Development Procedure
11/11/2013Fundamentals of Language Assessment
Reflect on the testing process
Learn from each exam Did it serve its purpose? Was it a valid and reliable test? Was it part of the students learning experience?
What future changes would you make?
-
8/14/2019 Testing and Assessing Final presentation.pdf
46/46
ReferencesAlastair Irons.(2008).Enhancing learning through Formative Assessment and Feedback. Routledge. Londonand New York
Anderson. L. W. (2003). Classroom Assessment. Lawrence Erlbaum Associates, Inc.
Bachman, L. F. (1990). Fundamental Considerations in Language Testing. Oxford applied linguistics. Oxford:Oxford University Press.
Brown, H. D. (2004). Language Assessment: Principles and Classroom Practices. New York: Pearson/Longman.
Brown, J. D. (1996). Testing in Language Programs. Prentice Hall Regents.
Gipps. C. V. (1994). Beyond Testing. The Falmer Press.Glenn Fulcher and Fred Davidson. (2007). Language testing and Assessment. Routledge. New York.
Heaton, J. B. (1988). Writing English Language Tests. New Edition. London: Longman,.
Hughs, Arthur. (1989).Testing for Language Teachers. New York: Cambridge University Press,
Kopriva. R. J. (2008). Improving Testing for English Language Learners. Routledge. New York.
Madsen. H. S. (1983). Techniques in Testing. Oxford University Press. New York.
Paul Black et al. (2003). Assessment for Learning: putting it into practice. Open University Press. England.
Peggy. L. Maki. (2004). Assessing for Learning. American Association for higher Education. Virginia.
Phil Race et al. (2005). 500 Tips on assessment. RoutledgeFalmer. London and New York.
Popham. W. J. (2003). Test Better, Teach Better: The Instructional Role of Assessment. Association forSupervision and Curriculum Development. USA.