1

Human Development

Md. Mehadi Rahman Roll: 16-589

18th BatchEvaluation and Educational Research(EER)

IERUniversity of Dhaka

Goal Free Evaluation• Goal-free evaluation (GFE) is any evaluation in

which the evaluator conducts the evaluation without particular knowledge of or reference to stated or predetermined goals and objectives.

• The goal-free evaluator attempts to observe and measure all actual outcomes, e ects, or impactsff , intended or unintended, all without being cued to the program’s intentions.

• GFE evaluator asks: What does the program actually do? Rather, what does the program intend to do?

Goal Free Evaluation

• “Merit is determined by relating program effects tothe relevant needs of the impacted population, (Scriven, 1991. p. 180).”

• A comprehensive needs assessment is conductedsimultaneously with data collection.

• “The evaluator should provide experiential accountsof program activity so that readers of the report can,through naturalistic generalization, arrive at theirown judgments of quality in addition to those theevaluator provides, (Stake, 2004 in Alkin, 2004, p. 215).

Goal Free Evaluation Model

Goal free evaluation was developed by Michael Scriven in 1972.

Michael Scriven(b. 1928)

• British born Australian philosopher• University professor, 50 years of experience• Bachelors and Masters from Melbourne University,

Australia• Ph.D. from Oxford University• Currently co-director of Claremont Evaluation Center• Exceeds 450 publications

Major Characteristics of Goal Free Evaluation

• Evaluator actively avoids information regarding program goals.

• Evaluator does not have preconceived goals in order to narrow the focus.

• Evaluator has minimal contact with staff or members of the program.

• Without information regarding goals, evaluator is more likely to see unanticipated effects of program.

Example

• An evaluator might be asked to evaluate the effectiveness of an adult basic education (ABE) project housed within the program of a local adult learning center (ALC). Also housed in that program are workplace literacy, welfare-to-work, and adult computer literacy projects. Clients of the adult learning center may participate in any or all of these programs. Thus it would be difficult to isolate the results of just one project's activities. A goal-free evaluation would examine the overall results for the clients of the ALC program, which would be more meaningful than individual evaluations of each project.

When to use Goal Free Evaluations

• When stakeholders want: Information about program outcomes, both intended

and unintended. Critique not focused on the program goals.• When evaluators: Have no knowledge of program goals, intentionally

or unintentionally Want to identify the effect of a program from data

collection, observations, and interviews

Needs and Nature of Goal Free Evaluation

• Goal free evaluation needs1) Access to program/project participant2) Access to all data3) Time• Nature of Goal free evaluation A goal free evaluation tends to be qualitative in nature.

That is, this evaluation is one of discovery. By interviewing participants in an unstructured interview,

the evaluator begins to identify outcomes. Several interview may be necessary. Participant

observation may, if possible, be necessary.

Methodologies • Scriven (1991) claims GFE is methodologically neutral,

which means that it can be used or adapted for use with several other evaluation approaches, models, and methods as long as the other approaches do not mandate goal orientation.

• Determine what effects this program had and evaluate them whether or not they were intended.

• Evaluate the actual effects against a profile of demonstrated needs

• Determine if what occurred can logically be attributed to the program or intervention.

• Determine the degree to which the e ectff is positive, negative, or neutral.

Methodologies • Notice something that everyone else overlooked over all

perspective.• Do not be under the control of the Management. Choose

the variables of the evaluation independently. • The key to goal-free evaluation is to have an evaluator

enter the field and try to learn about a program and its results inductively and without being aware of the specific objectives of the program.

• Note that GFE approach is useful as a supplement to the more traditional goal-oriented evaluation.

• Goal free evaluation is done by a separate evaluator, who collects exploratory data to supplement another evaluator’s goal-oriented data.

Critical Elements of Goal Free Evaluation

• Goal free evaluation demands that the evaluator have good knowledge of the subject of evaluation.

• Evaluator competence is a major issue. Included in this element is the requirement that the evaluator be free of bias.

Implementation Techniques for Goal Free Evaluation

• Ask questions and devise ways to find answers.• Determine what the evaluation will do and how it will benefit

stakeholders• Select appropriate methodologies for gathering data• Interviewing and interviewing again, Focus groups Development of emergent themes Reflective feedback Identify key and critical issues Forming theories Unexpected outcomes (Strength of Goal free evaluation)• Draw out key issues• Provide appropriate feedback and constructive criticism

Comparison between GFE Goals and Stated Goals

Comparison between GFE Goals and Stated Goals

Benefits of Goal Free Evaluation

Goal-free evaluation benefits are based on:

1. controlling goal orientation-related biases,2. uncovering side e ects,ff3. avoiding the rhetoric of “true” goals,4. adapting to contextual/environmental changes 5. aligning goals with actual program activities and

outcomes, 6. supplementing GBE

1. Controlling goal orientation-related biases:• Through reducing interaction with program sta and by ff

making the evaluator blinded from the program’s predetermined goals or objectives, GFE is less vulnerable to some of the social biases.

• Goal-free evaluation o ers fewer opportunities for evaluator ffbias in attempts to satisfy the evaluation client because the evaluator is therefore unable to determine ways of manipulating in the evaluation client’s favor.

2. Uncovering side e ects:ff• Goal-free evaluation can benefit foundations and their

programs because it is more likely to identify unintended positive and negative side e ects ff simply because the method allows for and encourages a broader range of outcomes as well as unanticipated outcomes

3. Avoiding the rhetoric of “true” goals:• Goal-free evaluation avoid the difficult rhetorical and often contaminating task in

traditional evaluations of trying to identify true current goals and true original goals, and then defining and weighting them.

• Historically, goals were embedded in professional mode, current jargon, or lists of priorities where “the rhetoric of intent was being used as a substitute for evidence of success” (Scriven, 1974, p. 35)

• The obvious issue is that when goals are poorly founded, the goal-based evaluator will miss critical e ects that may be detectable to the goal-free evaluator. ff

4. Adapting to contextual/environmental changes:• GFE can be adapted to the periodic changes in consumer needs, program

resources, and program goals. Consumers, programs, foundations, and their environments are dynamic.

• The goal-free evaluator can continue inquiry when a program’s goal changes, as long as changes in goals or objectives are reflected in the program’s actions and outcomes, the goal-free evaluator recognizes and records these e ects. ff

5. Aligning goals with actual program activities and outcomes:• The goal-free evaluator finds outcomes that are attributable to the program

intervention and renames these outcomes operating goals. All operating goals, therefore, have potential to become an official program goal or objective.

• GFE can be useful in aligning a program’s goals with its actual activities and performance, potentially resulting in a broader, more comprehensive list of criteria for judging a program’s merit and a more thorough examination of a program’s outcomes.

6. Supplementing GBE(Goal-based Evaluation):• An evaluation may begin goal-free and later become goal-based using the goal-free

data for preliminary investigative purposes; this ensures that the evaluator still examines goal achievement.

• The findings from the GFE can be used as baseline information for subsequent GBEs. Another example of GFE informing GBE is when GFE is used as a complement to GBE. A GBE and GFE can be conducted simultaneously by di erent evaluators.ff

Uses of Goal Free Evaluation

In School :• To evaluate programs targeting pre-requisite skills,

where success may impact more than initial skills, such as:

Reading fluency Mathematical operations and reasoning Writing skills Problem solving or critical thinking skill

Uses of Goal Free EvaluationIn Districts/Country:• To evaluate resource allocations, such as: Staffing ratios at primary, secondary, and high schools as

well as district office Budget expenditures Technological systems and access to information • To evaluate policy implementation, such as: Grading practices Attendance Rules Special programs and Services• To evaluate curriculum, such as: Required knowledge and skills

Criticism

• There is a chance that some of the most important effects will be missed.

• The model failed to come to grip with the question of what effects to look at, and what needs to be assessed. This eventually led Scrivens to admit that goal-free evaluation was best used as a supplement to goal-based evaluation.

• This approach can only lead to poor planning. • Goal-free evaluation is seen as a threat by many program

designers

Criticism• Some critics have faulted Scriven for not providing more

explicit directions for developing and implementing the goal-free model; as a consequence, it probably can be used only by experts who do not require explicit guidance in assessing needs and detecting effects.

• While it may be a very useful theory, it is not necessarily a practical model. Indeed, few cases of Goal-free Evaluation have been documented.

• Critics of Goal-free Evaluation consider the term ‘goal-free’ evaluation to be a misnomer. The evaluator does not get rid of all goals, but replaces the goals of the project staff with more global goals based on societal needs and basic standards of morality.

Reference

• https://www.wmich.edu/sites/default/files/attachments/u58/2014/cafe-Youker-Fall-051.pdf

• http://scholarworks.gvsu.edu/cgi/viewcontent.cgi?article=1182&context=tfr

• http://academlib.com/8293/management/goal-free_model• http://nsfconsulting.com.au/goal-free-evaluation/• https://www.youtube.com/watch?v=8lB9MiVxNT0• https://www.youtube.com/watch?v=__0yWU1Sh-w• https://www.youtube.com/watch?v=1uRdbnTpJaI

Thank you all