7/30/2019 EDTECH 505 - Week 7

1/18

Week 7 Parts D, E and FBy Jamie Doiron

7/30/2019 EDTECH 505 - Week 7

2/18

Thoughts on The UserFriendly Handbook for

Mixed Method EvaluationPart D

7/30/2019 EDTECH 505 - Week 7

3/18

Clear, easy to follow project description Concise goals Do goals stop short?

What about improving test scores of elementarystudents enrolled in classes?

Hard to track, but it would prove the effectiveness ofthe program downstream.

Clear difference between formative andsummative evaluation

Chapter 2 Reactions

7/30/2019 EDTECH 505 - Week 7

4/18

Chapter 2 Realizations

An evaluation may contain both formativeand summative evaluations

The design does not have to be 100%

complete in the RFP Project goals and description are separate

items

7/30/2019 EDTECH 505 - Week 7

5/18

Chapter 6 Reaction

A lot of items were undetermined prior to theRFP being accepted and the evaluationstarting

Budget Timeline Resources Evaluation plans

Excellent alignment between project goals,questions and stakeholders Alignment between questions, data sources

and collection methods

7/30/2019 EDTECH 505 - Week 7

6/18

Chapter 6 Realizations

Both quantitative and qualitative data areacceptable for an evaluation Probably best to have both instead of just one

Prioritization is required to achieveevaluation goals on schedule and withinbudget

A definitive schedule for data collection

helps keep the evaluation on track.

7/30/2019 EDTECH 505 - Week 7

7/18

Thoughts on Chapter 5 of

The Course TextPart D

7/30/2019 EDTECH 505 - Week 7

8/18

Distinction between research and evaluationmodels is confusing

Discrepancy model could be used for many

things because all systems areinterconnected - nothing exists in vacuum

Goal-free model measures effectivenessagainst a program's customers or users

Transaction model provides lots ofimmediate feedback to the program beingevaluated

Decision making model is for analyzing

Chapter 5 Reaction

7/30/2019 EDTECH 505 - Week 7

9/18

Chapter 5 Realizations

All evaluation models involve some sort of research Discrepancy model works best when standards exist To properly use goal-free model, evaluator must be truly

objective

Transaction model requires a good relationship betweenevaluator and program staff

Decision-making model is for long-term programs oroutcomes only

Systems analysis model is the most familiar to me as itis similar to what I did when working as a systemanalyst in an IT department

Goal-based method could apply to many types ofprograms

7/30/2019 EDTECH 505 - Week 7

10/18

Part D

Comparison of Chapter 5

and Chapters 2 & 6

7/30/2019 EDTECH 505 - Week 7

11/18

Chapters 2 and 6 of the internet readingsseem to use the goal-free evaluation model Measure effectiveness of program on the users of

the program All models discussed in all readings rely

heavily on data Data gathering is a large part of the

evaluation process for all models All readings discussed qualitative and

quantitative data All models require heavy stakeholder

involvement

Similarities

7/30/2019 EDTECH 505 - Week 7

12/18

Discrepancies

The internet readings dealt with only onescenario and model that used bothqualitative and quantitative data

The book compares and contrasts themodels but does not recommend one overanother

The scenario makes not mention of themodel used, only the process by which datawill be collected, analyzed and used

7/30/2019 EDTECH 505 - Week 7

13/18

Which Evaluation ModelBest Suits My Evaluation

Project?Part E

7/30/2019 EDTECH 505 - Week 7

14/18

Discrepancy model

Design Installation

Process Product Cost-benefit analysis

Compare to performance standards

My Evaluation Model

7/30/2019 EDTECH 505 - Week 7

15/18

Reasoning

Evaluation question: Will Google Sites meet theneeds of my school district and function as anLMS?

Compare google sites capabilities to theirneeds, which function as "performancestandards"

There is a cost involved Input from many stakeholders is required Determine differences between what is and

what should be

7/30/2019 EDTECH 505 - Week 7

16/18

Part F

Custom Assignment

Design

7/30/2019 EDTECH 505 - Week 7

17/18

Create a calendar for the entire evaluationthat cross references the data collectiontimeline, stakeholders needed, type of data

being collected, and collection methods.

Assignment Description

7/30/2019 EDTECH 505 - Week 7

18/18

My submission

Click the link below to be taken to an interactive googlecalendar. I added appointments that correspond to allnecessary meetings. If you click the appointment and viewthe details, you can see the questions from the scenario

that would be asked at those meetings. This could be usedby the evaluator as a timeline and scheduling tool forinterview and evaluation activities, as well as to keepstakeholders informed of the data that would need to be

gathered and the questions that would be discussed atinterviews. Appointments could be made right from theevents as well and sent to the stakeholders' emails. Thehypothetical evaluation and events begin in May 2013.

Click here for calendar.

http://www.google.com/calendar/embed?src=u.boisestate.edu_fnf41affqn6qjdpueu1fusi794%40group.calendar.google.com&ctz=America/New_Yorkhttp://www.google.com/calendar/embed?src=u.boisestate.edu_fnf41affqn6qjdpueu1fusi794%40group.calendar.google.com&ctz=America/New_York