ED 100 971

AUTHORTITLEINSTITUTION

SPANS AGENCY

PUB DATENOTE

!DRS PRICEDESCRIPTORS

IDENTIFIERS

DOCUMENT RESUME

88 TM 004 088

Saif, Philip S.On Evaluation: For Members of Boards of Education.Capitol Region Education Council, West Hartford,Conn.Bureau of Elementary and Secondary Education(DREW /OED, Washington, D.C.7421p.; For related documents, see TM 004 087, 092, and110

MF$0.75 HC$1.50 PLUS POSTAGE*Boards of Education; Decision Making; *Evaluation;Objectives; *Personnel Evaluation; ProgramEvaluation; Self Evaluation; Teacher EvaluationElementary Secondary Education Act Title III; ESEATitle III; 4Project Evaluation

ABSTRACTThe definition and purposes of evaluation are

explained in this booklet addressed to members of boards ofeducation. Evaluation provides information to the school boa-denabling them to make decisions about goals, personnel,administrators, and programs. Some guidelines are offered regardingresponsibility in personnel and student evaluation. (SM)

U.S. DEPARTMENT OF HEALTH,EDUCATION & WELFARENATIONAL INSTITUTE OF

EDUCATIONTHIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROMTHE PERSON OR ORGANIZATION ORIGINMING IT POINTS OF VIEW OR OPINIONSSTATED DO NOT NECESSARILY REPRESENT OFFICIAL NATIONAL INSTITUTE OFEDUCATION POSITION OR POLICY.

.PROJECT EVALUATION

Capitol Region Education Council

ON EVALUATION:

FOR MEMBERS OF BOARDS OF EDUCATION

Philip S. Saif

CO(X)

C)

OC) This booklet has been prepared with

PEE.S.E.A Title III funds

r- 1174

PREFACE

This booklet has been prepared bythe Capitol Region Education Councilto provide Boards of education withsome basic information about evalua-tion in education.

The contents of this bookletfocus more on the "What" and "Why"of evaluation, than on the "How."There are many materials at CRECwhich provide more detailed infor-mation on the process involved incarrying out an evaluation. Con-sultant help is also available atCREC.

The topic of evaluation is com-plex. A booklet of this nature isnot meant to provide a comprehen-sive picture of evaluation. Thisbooklet is concerned with some ques-tions on evaluation raised by someBoard members.

Readers needing or wanting moredetailed information are referred tothe CREC Evaluation Center.

To the Members of Boards of Ed-ucation who responded to the Center'sinvitation, to those who read themanuscript and reacted, and to LarryBenedict, who helped with his discus-sions, I would like to express mythanks.

Philip S. Saif, DirectorProject Evaluation

ii

The concept of evaluation is acentral concern to everyone involvedin education. Educators have alwaysbeen concerned with these five pointsof evaluation: ?hi to evaluate, Whatto evaluate, When to evaluate, How toevaluate and Ito evaluate. TilFdi-tionally, the concept of "evaluation".has been used interchangeably withthat of "testing" and everyone isfamiliar with testing, beginning withthe tests he took in first grade andcontinuing throughout his schoolcareer.

Recently, however, many educa-tors have begun to feel that simpletesting and test scores are notsufficient to meet education's needsor to make educational decisions.This booklet will focus on this con-cept and explore in more detail someof the topics of evaluation.

The purpose of Evaluation

Evaluation is done to collectspecific information about a parti-cular enterprise to facilitatedecisionmaking to enable the oper-ation of that enterprise to meet itsgoals. For example, an evaluationmight be undertaken to see if an ex-perimental high school geographyprogram is meeting its stated goals

1

which are: (a) to teach more geog-raphy skills, (b) to teach map skillsto slow learners, (c) to save money.When the program is in operation, in-formation (data) is systematicallycollected to see if the program ismeeting its goals. If the geographycourse is supposed to teach mapskills to slow learners, i4 Lt teach-ing map skills to slow learners?

Once this information is col-lected, a decision or set of deci-sions can be made about the program.If the data show that the slow learn-ers have mastered the requiredskills, a decision might be made tokeep the program. If it were a pilotprogram, a decision might be made toexpand it. Conversely, if the pro-gram failed to meet the expectationsset for it, a decision might be madeto modify it, or drop it entirely.

The collection of informationto make decisions about a particularprogram, a staff, an administrator,or even a School Board, is the pur-pose for doing any evaluation. Sys-tematic evaluation allows a decisionmaker or decision-makers (e.g., aSchool Board) ..to base decisions ondata rather than on personal feel-ings, intuitions, or vague andgeneral reviews by others.

Evaluation can provide data to aSchool Board facing the following

2

questions:

1. About an administrator: Is theadministrator doing the job heis supposed to be doing? Is thejob being done well? Is the jobbeing fulfilled in a manner con-sistent with the goals of theSchool Board; of the community?Is the job being done the waythe Board wants it to be done?

2. About the Board itself: Is theBoard doing the job it is sup-posed to be doing? Is it doingits job well? Is it accomplish-ing what it wants? Is it simplyacting as a group? Is it opera-ting with consistency? Has theBoard identified the job speci-fication of the Superintendent?

3. About programs: Is a given pro-gram meeting its goals? Does anexperimental program need alittle more, a lot more, or nomore support in order to work?Should the program be kept? Ex-panded? Modified?

ITilLNPOP!LPtqM414449A ineach (17timtheexalita.ove isto collect data which will help yieldbetter decisions. In these cases theBoard is the decision - maker, and ineach case particular information is

3

needed to make decisions. Informa-tion concerning an administratorwould not be the same as informationregarding a particular program.Therefore, data must be collectedgiving precise information concerningpoints determined to be important.

Some Basic Elements of Evaluation

Whether evaluation is to collectdata about programs, personnel, orsome other aspect of an educationalsystem, the evaluation will have cer-tain basic elements. These elementscan be phrased as questions, and theprocess of answering these questionscomprises some of the basic steps ofevaluation.

Step 1

Why its it nece.wity to haveevatuatton?

Any Board of education is makingdecisions. These decisions concernbudget, personnel, and programs. Inorder that a Board can make soundjudgments, relevant data must be pre-sented.

4

Ste

What i4 to be evaluated?

A clear definition of the enter-prise to be evaluated must be given.Is it a new program, the Superinten-dent; or a set of materials? Often,when asked what is to be evaluated,an educational decision-maker may re-spond, "the aehoot," or "the etemen-tatty p4og4am," or "the Engti4h wo-g/cam.' A school, or an elementaryprogram or an English program arecomprised of many different elements:materials, students, personnel, etc.A clear definition of what is to beevaluated becomes the crucial start-ing point.

Step 3

FOIL whom L4 the opekatton anentetptaAte to be evaluated?

For any given enterprise, thereare a number of decision-makers, eachmaking different kinds of decisions.A Board makes different decisionsabout the Superintendent than does afaculty; therefore, each would wantdifferent data for making their re-spective decisions. If the enter-prise were a new geography course, anevaluation could be done from the

5

viewpoint of several differentdecision-makers: the teachers teach-ing it, the students taking it, theprincipal in whose building it isbeing implemented, the Board whoauthorized funds for it.

An evaluation done for theteachers of the course would providedata for use in making instructionaldecisions. However, these data wouldnot necessarily be useful for aSchool Board, which would be makingdecisions on a different level. Mosttraditional evaluations do not pro-vide for different decision--makerswith different needs. Therefore,part of the evaluation process shouldbe the identification of the variousdecision-makers and a determinationof the primary decision-maker forwhom the evaluation will be made.

what aae the goats 04 the en-tem:044:6e?

Once it has been determinedwhat is to be evaluated and forwhom it is to be evaluated, it isnecessary to determine the goalsheld for the enterprise. Goal set-ting becomes an important part of theevaluation process.

6

BEST COPY AVAILABLE

A math program cannot be evalua-ted unless the goals held for it areclearly specified. A Superintendentcannot be evaluated unless the goalsof his job are clearly stated. Infact, a Board cannot evaluate itself,or determine if it is working satis-factorily, if it has not clearlystated its goals.

Without identifying goals, it isliterally impossible for a elhoolBoard to evaluate effectively andsystematically, its schools, its pro-grams, its personnel or its own oper-ation. One cannot determine whethersomething, or someone is doing well,if goals have never been identifiedand stated clearly and correctly.

Unfortunately, goals tend to beexpressed as vague concepts: "goodeitizen4," "peat Supenintendent,""good 4ehoot 4yestem," Undtvtdadt-ized teautng." These goals are toogeneral for meaningful data collec-tion. The decision-makers must takethese goals and make them morespecific.

Decision-making by the Board isgreatly hindered by the failure toidentify and establish priorities onits goals. Without priorities,decision-making becomes a process ofindividuals acting intuitively and

I

inconsistently, leading to an atti-tude on the part of the community orthe school personnel of, "Wett, whatwitt they do nextf" In the final in-stance, a Board is accountable to thecommunity, but if the Board never es-tablishes concrete objectives, it cannever say to the community that ithas accomplished its purpose.

It should be emphasized that theBoard must actually identify its owngoals. It is the Board's right, andindeed responsibility, to determineits goals as well as the uses towhich data collected will be put.

S,ten 5

When its evacuation done?

Evaluation is a continuous pro-cess. Evaluation that begins andstops at a particular period of timeis not a complete evaluation. An im-portant part of the evaluation isthe feedback to the original goals orobjectives to check whether or notsuch goals should be modified, de-leted, or left as they are. Evalua-tion must be a continuous process.

8

S,tea 6

How 4.4 evatuation pentimmed?

This booklet is not intended toelaborate on this point. However,the various methods used in evalua-tions include: scales, lists, ques-tionnaires, opinionnaires, tests,video tapes, audio tapes, etc. Thisdepends upon what is to be evaluated.

Ste

What 4.6 done with the data?

Decision-making. Once data arecollected and presented a4 objec-tivety a4 po44ibte, the decision-maker(s) (here they are the membersof the Board) will make their de-cision(s). The decision(s) will bebased upon the collected information.

Caution must be tised related todecision-making. For example, ifBoard members came to a conclusionthat the Superintendent should leave,t.is should not be interpreted thatthe educational programs he intro-duced are bad. Each component of thesystem must have its identity, evenif there is an obvious relationship.

The reverse is also true; i.e.,a program that proved to be or no

9

BEST COPY IIIIMARE

value in meeting the goals of educa-tion in a particular community doesnot necessarily mean that the Super-intendent is bad (at fault). Appro-priate evaluation of personnel is amajor task of a School Board.

1. lagetintendeAt Evaluation

How to evaluate the Superinten-dent is one of the common problemsfacing Board members. One of theproblems in Superintendent evaluationis that in too many cases goals forthe Superintendent are not clearlyand specifically set forth.

When asked how he evaluates hisSuperintendent, one Board member re-plied that his yearly evaluation ofthe Superintendent is: "Gee, 1 thtnkyou axe doing a tkemendous job. I

want to vote to /anew yowl, contuct."

His "evaluation" is based onfeelings and emotions, or simply amatter of likes and dislikes, and heexpresses his frustration with thisapproach. Decisions are often basedon criteria such as being a neatdresser, being charming, not "rock-ing the boat." Needless to say, thisis not a very systematic evaluation.

10

Usually there are more sub-stantive goals held for the Superin-tendent. Such goals or criteriashould be clearly stated, and every-one involved should be aware of them.Goals and standards of performanceshould be set by the Board, with theSuperintendent.

2. Pecsonnet Evaluation

Personnel evaluation is anothercrucial element in the educationalevaluation process. Like Superin-tendent evaluation, it has certainbasic elements: "personnel" to beevaluated; goals for the personnel;objectives to meet the goals; and adecision-maker for whom the evalua-tion is made.

Two issues arise, both from theperspective of Board members andfrom the perspective of teachers:

(a) Who is responsible for per-sonnel evaluation, and

(b) How is evaluation differentfrom "rating."

(a) Responsibility. Ultimately, inany pubffb school district, the

11

School Board is responsible for per-sonnel evaluation. However, thereare other levels of responsibilityinvolved in carrying out such evalua-tion:

(1) The Superintendent: The Boardwill probably The directly re-sponsible for evaluation ofthe Superintendent.

(2) Principals and administrators:These persons are probab yedirect responsibility of theSuperintendent. It is reason-able for a Board to expect tobe told the criteria which theSuperintendent uses to evalu-ate principals and other ad-ministrators. The SchoolBoard, as a group, should knowthe criteria used to evaluateadministrators. This does notmean that the Board Eiiai-TEget involved in the evaluationof each staff member. It doesmean that they should takesome interest in establishingthe goal setting phase of theevaluation process.

(3) Teachers: The Board should beMinn to participate in theestablishment of criteria tomake teacher evaluation rel-evant to the goals of the edu-cational system. Failure to

12

do this invites a good deal ofinconsistency into a system.This becomes obvious at con-tract time of year when aBoard is usually presentedwith a list of names for ten-ure, rehiring or dismissal.Some Board members have ex-pressed frustration with sucha process. They have raisedsuch questions as:

What mite/Lig weave used tomake these tecommendations?

Aare they consistent with thegoats o 6 the school diet/L.1,a?

Ake they conSistent withptans?

Ate they consistent with whatthe Boated wants?

Each of these questions shouldbe seriously examined andapproved by the Board beforean evaluation takes place.

(b) Evaluation as "ratin ": One ofthe most common compla nts raised byteachers and administrators alike isthat personnel evaluation is littlemore than a popularity contest, a"rating" based, at best, on veryvague criteria. This implies that

13

the evaluation is not tied to anygoals, but simply provides an indexof comparison with respect to otherteachers in the system, or with re-spect to the experience of the eval-uator. However, ratings could be apart of an on-going and systematicdiagnostic evaluation process.

Evaluation must be an on-goingprocess. It cannot occur once ortwice a year and have any real mean-ing. It must provide data from whichdecisions can be made in order to im-prove instruction. There must not beonly one evaluation; rather a seriesof dirrerent data collection tech-niques must be used by a variety ofdifferent people, possibly includingthe teacher himself.

Answers to three questions --What 4.4 to be evaluated, on whom,and what Vie the goats -- form thefirst part of an evaluation. It isa crucial part and often overlooked.Once these three steps have been com-pleted, data collection begins.

How are the data generated?

Once goals have been defined,there must be a design for data col-lection procedures, and a report tothe decision-maker. Any evaluationwould have these basic elements in

14

common. Judgments about the enter-prise, based on data, can then bemade by the decision-maker. Sincethis is not a "how to" booklet, thistopic will not be presented in detailhere.

A related issue has been raisedby some Board members: Now do etec-tiont which change the make-up oi aBoan.d qiect the educationat goect4hetd by the Some

Granted, a School Board electioncan Featly change the make-up of thecommittee: Two points should be madehowever:

Cl) Certain goals are unlikely tochange, even with a new elec-tion. Such goals might in-clude: wanting children to beable to read, write, do basiccomputation, perform certainphysical activities. In otherwords, even if a completelynew committee is elected, somegoals will remain the same.Once these goals have beenclearly established, they arelikely to remain unchanged,regardless of elections.

(2) Some goals or priorities ongoals are bound to change,even if Board elections do notchange the make-up of the

15

committees. The Board shouldhave a procedure for a per-iodic review of goals, regard-less of change. After all,educational goals are likelyto change to a certain extent,simply because of changingtimes and attitudes.

Should the Board members be involvedin student evaluation.

The Board should confine itselfto drawing the policy, and leave thedetails to the professional educators.For example, the Board, upon gatheredinformation, finds that usingCriterion-Referenced Tests will be ofmore value than Norm-Referenced Tests.Therefore, a policy to use Criterion-Referenced Tests should be writtenand the Superintendent charged withimplementing it.

Conclusion

Evaluation is meant to servedecision-makers by providing data tohelp them improve their decision-making. It is meant to help educa-tors determine how well they aredoing something with the purpose ofimproving it; be it programs, per-sonnel, student learning, or Board

16

operation. Part of the overallpurpose of improving education is im-proving its various parts or compo-nents.

Given the increasing pressureson, and demands for resources; giventhe need for program and personnelaccountability in the face of limitedresources; evaluation has become anincreasingly important decision-making tool.

17