developing your evaluation toolkit

35
DEVELOPING YOUR EVALUATION TOOLKIT Focus on Learning 2013 Paula Buskard and Sandy Odrowski

Upload: vince

Post on 26-Feb-2016

53 views

Category:

Documents


5 download

DESCRIPTION

Developing your Evaluation Toolkit. Focus on Learning 2013 Paula Buskard and Sandy Odrowski. Agenda . Session Outcomes Overview of what/why we evaluate Aligning evaluation methods to outcomes Evaluation Planning After the Evaluation: Debriefing Q & A. Session Outcomes. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Developing   your Evaluation Toolkit

DEVELOPING YOUR EVALUATION TOOLKIT

Focus on Learning 2013

Paula Buskard and Sandy Odrowski

Page 2: Developing   your Evaluation Toolkit

AGENDA

Session Outcomes

Overview of what/why we evaluate

Aligning evaluation methods to outcomes

Evaluation Planning

After the Evaluation: Debriefing

Q & A

Page 3: Developing   your Evaluation Toolkit

SESSION OUTCOMES

By the end of this session the participant will have the ability to:

Select suitable evaluation tools that are learner–centred and align to learning outcomes

Identify best practice assessment tools that prepare the instructor for writing great evaluations.

Identify ways to prepare the learner for evaluation.

Discuss ways to conduct effective post test debriefing for both learner and instructor.

Page 4: Developing   your Evaluation Toolkit

PLACE MAT ACTIVITY

Each member will takes a square/triangle and individually writes 3 questions/concerns they have about evaluation.

Page 5: Developing   your Evaluation Toolkit

RECORD THE GROUPS TOP 3 CONCERNS IN THE CENTRE

Page 6: Developing   your Evaluation Toolkit

EVALUATION –WHAT IS IT??

Page 7: Developing   your Evaluation Toolkit

FORMATIVE

diagnostic tool (Where are learners at? What have they got so far? Do I need to go over something again?)

help learners take stock of and assess their own progress towards learning goals

Page 8: Developing   your Evaluation Toolkit

SUMMATIVE:

assess a learners achievement of meeting the learning outcomes

determine grades

Page 9: Developing   your Evaluation Toolkit

AN ANALOGY OF LEARNING...9

Formative Summative

Page 10: Developing   your Evaluation Toolkit

SOMETHING TO THINK ABOUT

What is Needed to know (Learning Outcome)

What is Nice to know(Filler)

Page 11: Developing   your Evaluation Toolkit

WHAT TO EVALUATE?

The Course Learning Outcomes describe clearly what

learners will know and be able to do at the end of the course.

They are performance-based and

results oriented.

Page 12: Developing   your Evaluation Toolkit

THE GOLDEN CHAIN

Page 13: Developing   your Evaluation Toolkit

SNOWBALL EXERCISE

Each group move to a posted Learning Outcome

Take 2 minutes (I will call time) to identify evaluation methods that could be used to measure learning.

When time is called, move to the next Learning Outcome and repeat.

Once each group has had the opportunity to identify evaluation methods for each Learning Outcome let us know.

Share in large group.

Page 14: Developing   your Evaluation Toolkit

WHAT DID YOU LEARN TODAY

Working with a partner, fill in the Handout labeled Evaluation Strategy Matching Exercise.

Page 15: Developing   your Evaluation Toolkit

EVALUATION METHODSADVANTAGES AND CONCERNS

In your group:

Take a moment to discuss the Advantages and Concerns regarding the evaluation method listed on the flip chart paper at your table.

Record your findings and be prepared to discuss with the larger group

Page 16: Developing   your Evaluation Toolkit

EVALUATION PLANNING

Page 17: Developing   your Evaluation Toolkit

THE GOODS

“Will this be on the test?”

“What do I really have to know?”

“How do I answer this question?”

“Why did I get this question wrong?”

 

Page 19: Developing   your Evaluation Toolkit

FOUR POINTS TO CONSIDER:

1 – Measure what is most important rather than what is easiest to measure

2 - Be aware of the learning embedded in evaluation

3 – Understand the evaluator’s power

4 – Take care not to reduce the curriculum to only what is evaluated

Page 20: Developing   your Evaluation Toolkit

GUIDELINES FOR PLANNING

Start early (holidays, sequence, scaffolding)

Determine the number of items and time limitations

Construct the question- make sure that it measures learning and is linked to content objectives/outcomes

Edit the questions for errors in spelling, sentence structure, etc.

Page 21: Developing   your Evaluation Toolkit

GUIDELINES FOR PLANNING CONT’D…

Group the questions according to type and topic (make easier ones first)

Write directions (include marking scheme)

Have the answers “ready” for post test debrief

Review

Page 22: Developing   your Evaluation Toolkit

HOW TO WRITE A GOOD EVALUATION

Learner-Centered and Authentic Assessment Checklist

Developing Good Test Questions

Page 23: Developing   your Evaluation Toolkit

PREPARING YOUR LEARNERS

Page 24: Developing   your Evaluation Toolkit

THE GOLDEN CHAIN?

Develop a business plan for a small non profit organization.

Lecture and homework: practice multiple choice questions from the text

Evaluation: Final Exam: multiple choice.

Course Learning Outcome

Learning and practice methods

Evaluation Method

Page 25: Developing   your Evaluation Toolkit

THE GOLDEN CHAIN?

Develop a business plan for a small non profit organization.

Lecture: types of business and parts of a business plan/. Case studies

Student develop a business plan for a mock or real non profit organization

Course Learning Outcome

Learning and practice methods

Evaluation Method

Page 26: Developing   your Evaluation Toolkit

DEBRIEFING AND FEEDBACK

Page 27: Developing   your Evaluation Toolkit

TEACHER DEBRIEF

Exercise: In your small groups answer these 2 questions:

What could teachers consider after a test?

What could teachers do to help their learners after a test?

Page 28: Developing   your Evaluation Toolkit

TEACHER DEBRIEF: CLEAN UP

Did my learners understand what was asked on the test?

Were there questions that the learners missed that were the fault of the teaching? How will I remedy this?

Do the results indicate that the learners were prepared for the evaluation?

Page 29: Developing   your Evaluation Toolkit

TEACHER DEBRIEF: CLEAN UP CONT’D

• If I use this procedure again, what changes will I make?

• What other methods of evaluation might be more appropriate?

• Is this an evaluation that must be returned?

• Is there evidence of academic dishonesty? If so, what can I do about it (this time and in the future?)

Page 30: Developing   your Evaluation Toolkit

LEARNERS DEBRIEF

Quickly go over the test and provide the correct answers to all of the questions.

Discuss the outstanding questions – those that most students answered well, those that some may have got stuck on, etc.

Open the floor for focused discussions (learning opportunity).

After a reasonable time is spent on a class discussion, invite students to make appointments with you to discuss individual concerns.

24 hour rule

Page 31: Developing   your Evaluation Toolkit

EVALUATION WORKS BEST WHEN…

Provide clear, objective assignment guidelines

Provide samples, if possible, of desired product

Provide a marking rubric showing where the marks will come from

Tie evaluation strategy directly to the learning outcome being measured Domain/level

Page 32: Developing   your Evaluation Toolkit

ANY QUESTIONS/CONCERNS LEFT?

Page 33: Developing   your Evaluation Toolkit

ANY OTHER QUESTIONS?

Page 35: Developing   your Evaluation Toolkit

DO YOU WANT TO LEARN MORE?

Andrus, C and Bloor, D (2005) Teaching tools

Gross-Davis, B. (2001). Tools for teaching. San Francisco: Jossey-Bass

Fenwick T. and Parsons, J. (2000). The art of evaluation: A handbook for educators and trainers. Toronto: Thompson Educational Publishing

http://www.naacls.org/docs/announcement/writing-objectives.pdf

http://rubistar.4teachers.org/index.php

http://pareonline.net/getvn.asp?v=7&n=3