best practices in efl assessment
DESCRIPTION
Best practices in EFL AssessmentTRANSCRIPT
Best Practices in Classroom Assessment
By Mohamed Mahgoub, MA TEFL (AUC)
Workshop Schedule – Day 1
Time Topic Outcomes
07:45 – 09:15(90 minutes)
SMART Objectives
SMART objectives developed by subject-specific groups
09:15 – 10:45 (90 minutes)
Characteristics of effective assessment
Types of assessment
Assessment for Learning
Ways of maintaining validity and reliability in assessment tools are identified and described
Various types of formative and summative assessment tools are identified and described
Various strategies for applying Assessment for Learning are identified and planned
10:45: 13:45(90 minutes)
Classroom observations
Feedback (self & peers) of classroom applications of formative assessment (especially Assessment for Learning)
3Workshop Objectives By the end of this workshop, participants will be able to:
write SMART objectives for their lessons and activities addressing the three domains of learning and both HOTs (Higher Order Thinking skills) and LOTs (Lower Order Thinking skills) according to Bloom’s taxonomies
identify the characteristics of an effective assessment tool, especially reliability and validity
recognize the paramount importance of formative assessment ( as a core means of assessment for learning) besides using summative assessment
design and develop a variety of assessment tools and instruments satisfying various learners’ needs and variables (e.g. games, projects, tasks, quizzes, presentations, roles plays, and tests)
4Workshop Objectives
By the end of this workshop, participants will havedevelop tools that increase the objectivity of subjective assessment tools, e.g. essay and presentations, by developing rubrics
appreciate the role of assessment (especially formative) in supporting learning
engage their learners in choosing and developing a variety of assessment tools suitable for learner’s needs and learning objectives
What’s Assessment? .
- What do we assess? - Who assess what? - How do we assess?
Non-Traditional Teaching Style
Planning
Needs/Goals/
Objectives
Materials (realia,
authentic, published,
on-line)
Teaching Activities – Learning Activities
Learning product/
evidence
Program (planning, materials,
teaching & learning,
Assessment)
Evaluation
Learners (MIs, Learning Styles,
Learning needs, etc.)
Domains of Learning
Knowledge (Cognitive), Skills (Psychomotor), Attitudes (Affective) = KSA
Bloom’s Research: Domains & Taxonomies
Benjamin Bloom 1956
Cognitive Taxonomy
Curriculum Framework
Validity & Reliability = Quality Assessment
Validity Watch the video clip and
- Define validity
- Identify two types of validity
http://www.youtube.com/watch?v=iXaav7wA20Y
Reliability
Watch the video clip and - Differentiate between validity and reliability - Identify two types of reliability - Identify the relationship between validity and reliability
Key Ideas in Validity and Reliability for Teachers
http://www.youtube.com/watch?v=IF-oeuidRuU
How to Improve Assessment Validity (Nitko p 41)Category Criteria to be attained. Your assessment should…
Content representativeness and relevance (Content Validity)
Emphasize what you taught Represent school’s stated curricular content Represent current thinking about the subject
Thinking processes and skills represented
Require learners to integrate and use several thinking skills
Consistent with other classroom assessment
Yield patterns of results consistent with your other assessments of the class
Reliability and objectivity Provide each learner with several opportunities to demonstrate competence for each learning objective assessed
Fairness to different types of learners
Contain tasks that are interpreted appropriately by students with different backgrounds
Accommodate learners with disabilities Free of ethnic, racial, and gender bias
Economy, efficiency, practicality, instructional features
Require reasonable amount of time Represent reasonable use of learners’ class time
Multiple assessment usage Be used together with other class assessments for making important decisions
How to Improve Assessment Reliability In short In details
Lengthen the assessment procedure.
Give more time, use more questions, more observations, etc.
Broaden the scope of the procedure
Use procedures that assess all of the essential and important aspects of the target learning performance
Improve Objectivity Use a systematic, more formal procedure for scoring learners performance (rubric)
Use multiple markers (raters) (Inter-rater reliability)
Have more than one qualified person score each learner’s essay, performance, portfolio. Average the results
Combine the results from several assessments
Use a combination of results from several assessment methods (formal and informal) rather than a single assessment result
Teach your learners how to perform their best
Train your learners on performing in assessment situations
Match assessment difficulty to learners’ ability level
Use assessment tools that are not too easy or too difficult (within their Zone of Proximal Development-ZPD)
Types of Assessment 1. Formative Assessment
A. Conversations and comments from other teachers
B. Casual conversations with learners
C. Questioning learners during teaching/learning
D. Daily homework & classroom-based work (learning activities)
E. Teacher-made quizzes & tests
F. In-depth interviews of individual learners
G. Growth & learning-progress portfolios
H. Attitudes and values questionnaires
2. Summative Assessment A. Teacher-made quizzes and tests
B. Tasks focusing on procedures and processes
C. Product-oriented projects and tasks
D. Best works portfolios
E. Textbook-supplied quizzes and tests
F. Standardized achievement tests (Thanawya Aama, IGCSE, American Diploma)
Assessment of Learning & Assessment for Learning
Formative Assessment
- Principles of formative assessment
- http://www.youtube.com/watch?v=gXOMbXeQNNA&feature=related
- Formative Assessment Techniques
- http://www.youtube.com/watch?v=Zc67rq8XmtA&feature=related
- Formative Assessment in Classrooms
http://www.youtube.com/watch?v=D7XpUqNnKtI&feature=related
- Assessment for Learning Innovative Ways to check for understanding
http://www.youtube.com/watch?v=2C10oV0k3rE&feature=related
Assessment for Learning 1. Ensure you share the activity objective with your
learners in their own language
2. Work with them to identify success criteria (in achieving the objective)
3. Engage them in choosing the teaching and learning materials and resources
4. Either give them a demo on how to achieve the activity/task/learning Or Give them the task/activity and collaborate with them to achieve it (Deductive Vs Inductive) Inductive approach is more learner-centered; similar to Test Teach Test
Assessment for Learning 1. Enable them to Think, Pair, Share (More pair and
group work than front teaching and individual question and answer)
2. Encourage self-assessment and peer assessment
3. Encourage giving quality feedback (self, peer, then teacher if needed), focusing on positive, successful achievements then areas for improvement
4. Work with them to identify and collect evidence of learning
Checklist for Checking Quality of Short Answer Items
1. Does the item assess an important aspect of the unit’s instructional targets (goals)?
2. Does the item match your assessment plan in terms of performance, emphasis, and number of points?
3. If possible, is the item written in a question format?
4. Is the item worded clearly so that the correct answer is a brief phrase, single word, or single number?
5. Is the blank or answer space toward the end of the sentence?
Checklist for Checking Quality of Short Answer Items
6. Is the item paraphrased rather copied verbatim from learning materials?
7. If the item is in the completion format, is the omitted word an important word rather than a trivial word?
8. Are there only one or two blanks?
9. Is the blank or answer space in this item (a) the same length as the blank in other items, or (b) arranged in an appropriate column?
10. If appropriate, does the item (or the directions/instructions) tell the learner the appropriate degree of detail, specificity, precision, or units you want the answer to have?
11. Does the item avoid grammatical (and other irrelevant) clues to the correct answer.
Checklist for Checking Quality of True-False Items
1. Does the item assess an important aspect of the unit’s instructional targets (goals)?
2. Does the item match your assessment plan in terms of performance, emphasis, and number of points?
3. Does the item assess important ideas, knowledge, or understanding (rather than trivial, general knowledge, or common sense)?
4. Is the statement either definitely true or definitely false without adding further qualifications or conditions?
5. Is the statement paraphrased rather than copied verbatim from learning materials?
Checklist for Checking Quality of True-False Items
6. Are the word-lengths of true statements about the same as those of false statements?
7. Did you avoid presenting items in a repetitive or easily learned pattern (e.g., TTFFTT…, TFTFTF…)?
8. Is the item free of verbal clues that give away the answer?
9. If the statement represent an opinion, have you stated the source of opinion?
10.If the statement does not assess knowledge of the relationship between ideas, does it focus on only one important idea?
Checklist for Checking Quality of Multiple Choice Items
1. Does the item assess an important aspect of the unit’s instructional targets (goals)?
2. Does the item match your assessment plan in terms of performance, emphasis, and number of points?
3. Does the stem ask a direct question or set a specific problem?
4. Is the item based on a paraphrase rather than words lifted from a textbook?
5. Are the vocabulary and sentence structure at a relatively low and non-technical level?
Checklist for Checking Quality of Multiple Choice Items
6. Is each alternative (foil) plausible so that a student who lacks knowledge of the correct answer cannot view it as absurd or silly?
7. If possible is every incorrect alternative based on a common learner error or misconception?
8. Is the correct answer to this item independent of the correct answer of other items?
9. Are all of the alternatives homogeneous and appropriate to the content of the stem?
10.Did you avoid using “all of the above” or “non of the above” as much as possible?
11. Is there only one correct or best answer to the item?
Checklist for Checking Quality of Matching Exercises
1. Does the exercise assess an important aspect of the unit’s instructional targets (goals)?
2. Does the exercise match your assessment plan in terms of performance, emphasis, and number of points?
3. Within this exercise, does every premise and response belong to the same category of things?
4. Do your directions (Instructions) clearly state the basis you intend your learners use to complete the matching correctly?
5. Does every element in the response list functions as a plausible alternative to every element in the premise list?
Checklist for Checking Quality of Matching Exercises
6. Are there fewer than 10 responses in this matching exercise?
7. Did you avoid “perfect matching”?
8. Are the longer statements in the premise list and the shorter statements (names, words, symbols, etc.) in the response list?
9. If possible, are the elements in the response list ordered in a meaningful way (logically, numerically, alphabetically., etc.)?
10.Are the premises numbered and the responses lettered?
Checklist for Checking Quality of Essay
1. Does the essay assess an important aspect of the unit’s instructional targets (goals)?
2. Does the essay match your assessment plan in terms of performance, emphasis, and number of points?
3. Does the essay require learners to apply their knowledge to a new or novel situation?
4. When viewed in relation to other items on the test, does this item (the essay) contribute to covering the range of content and thinking skills specified in your assessment plan?
5. Is the prompt focused? Does it define a task with specific directions/instructions, rather than leave the assignment so broad that virtually any response can satisfy the question?
Checklist for Checking Quality of Essay
6. Is the task defined in the prompt within the level of complexity suitable to your learners?
7. To get a good mark on the item, is the learner required to demonstrate more than recall of facts, definitions, lists, generalizations, etc.?
8. Is the prompt worded in a way that leads all learners to interpret the assignment in the same way you intended?
9. Does the wording of the prompt make clear to the learners all of the following:
A) Magnitude or length of the required writing?
B) Purpose for which they are writing?
C) Amount of time to be devoted to answering this item?
D) Basis on which their answers will be evaluated?
10. If the essay prompt asks learners to state and support their opinions on controversial matters, does the wording make it clear that the learners’ assessment will be based on the logic and evidence, supporting their arguments, rather than on the actual position taken or opinion stated?
Checklist for Checking Quality of Performance Tasks
1. Does the task focus on an important aspect of the unit’s learning goals?
2. Does the task match your assessment plan in terms of performance, emphasis, and number of points (marks/weight)?
3. Does the task actually require a learner to do something (performance) rather than requiring only writing about how to do it, or simply to recall or copy information?
4. Do you allow enough time so all of your learners can complete the task under your specified conditions?
5. If this an open-response task, do your wording and directions make it clear to learners that they may use a variety of approaches and strategies, that you will accept more than one answer as correct, and that they need to fully elaborate their response?
6. If the task is intended to be authentic or realistic, do you present a situation that your level of learners will recognize as coming from the real world?
Checklist for Checking Quality of Performance Tasks
7. If this task requires using resources and locating information outside the classroom, will all of your learners have fair and equal access to the expected resources?
8. Do your directions and other wording:
A. define a task that is appropriate to the educational maturity of your learners?
B. lead all learners, including those from diverse cultural and ethnic backgrounds, to interpret the task requirements in the way you intend?
C. make clear the purpose or goal of the task?
D. make clear the length or the degree of elaboration of the response expected?
E. make clear the bases on which you will evaluate the response to the task?
9. Are the drawings, graphs, diagrams, charts and other task materials clearly drawn, properly constructed, appropriate to the intended performance and in good working order?
10. If you have learners with disabilities in your class, have you modified or adapted the task to accommodate their needs?