6433382

7/28/2019 6433382

1/34

JOURNAL OF PUBLIC RELATIONS RESEARCH, 10(2), 103-135Copyright 1998, Lawrence Erlbaum Associates, Inc,

Demonstrating Effectiveness in PublicRelations: Goals, Objectives, andEvaluationLinda Childers Hon

Department of Public RelationsCollege of Journalism and Com munications

University of Florida

F*ublic relations planning and evaluation were explored among 32 practitioners and10 top executives. Prac titioners said their goals reflect the priorities of their institution.The CEOs believed public relations' ultimate aim is comm unicating the image of theorganization. Responses showed many practitioners conduct informal evaluation,whereas only a few conduct formal evaluation. This research suggests pub lic relationsplanning and evaluation are becoming more systematic but are still constrained bylack of resources and difficulty.

Evaluating the effectiveness of public relations continues to be a topic of criticalimpottance to practitioners atid scholars. As pressures for accountability mount,practitioners increasingly must demonstrate that public relations activities helpachieve meaningful goals for their organization or clients (Johnson, 1994; Kirban,1983; "Measurement Driving More PR Programs," 1996).

Academ ics have long extolled tbe link between demonstrating public relationseffectiveness and evaluation. As a result, much attention has been devoted toanalyzing public relations measurement techniques (see Broom & Center, 1983;Broom&Dozier, 1983,1990; Cline, 1984;Dozier, 1984,1985;J.E. Grunig, 1977b,1983; J. E. Grunig & Hickson, 1976; Suchman, 1967; W right, 1979).

7/28/2019 6433382

2/34

104 HON

Although definitions of effectiveness in public relations abound (Hon, 1997),the most predominant theme in scholarly literature and the trade press is thateffective public relations occurs when communication activities achieve commu-nication goals (in a cost-efficient manner). Implicit in this model is that publicrelations goals are derived from the overall organizational mission, goals, andobjectives. Thus, through public relations evaluation, practitioners can dem onstrateeither directly or indirectly public relations' role in organizational goal achieve-ment.Given the pervasiveness of this model, it is ironic that so little research has beenconducted on goal setting and evaluation from practitioners' points of view. The

scant bit of research about goal setting in public relations is almost entirelyprescriptive-emphasizing what practitioners should be doing rather than illumi-nating what they really are doing and why. Research on public relations evaluationis much more voluminous than that on goal setting, and it is both descriptive andprescriptive. Yet, this body of work too could be enriched by showcasing practi-tioners' views of the complex realities they face when trying to evaluale theeffectiveness of their public relations efforts.

PURPOSE AND SIGNIFICANCE OF THE STUDY

The purpose of this research then is to explore issues surrounding goals, objectives,and evaluation in pub lic relations. The ultimate intent is a better understanding ofthe opportunities and constraints practitioners face in demonstrating the value ofpublic relations. This study is a foUow-up to earhei research (Hon, 1997) thatexplored practitioners' and CEOs' definitions of effectiveness in public relationsand their perceptions of the value public relations brings to clients and organiza-tions.The significance of investigating effectiveness in pub lic relations is great. Fewscholars or practitioners would dispute that the long-term viability ofthe publicrelations function is directly linked to a better understanding of public relations'value-added contribution (see C. Campbell, 1993; Geduldig, 1986; J. E. Gruniget ai., 1992; J, E. Grunig et al., 1991; Macnamara, 1992; "Measurement Popularin U.S.," 1995; Pritchitt, 1994; "Setting Benchmarks Leads," 1993; Strenski,

1981; Weiner, 1995; Winokur & Kinkead, 1993). Effectiveness measures that

7/28/2019 6433382

3/34

DEMONSTRATING EFFECTIVENESS IN PR 1 0 5

RESEARCH ON PUBLIC RELATIONS GOALS ANDOBJECTIVES

The assum ption found in public relations literature is that goals are general guidelinesor a framework for the public relations department, whereas objectives are thespecific outcomes desired from programs (J. E. Grunig & Hunt, 1984). Explainingthe distinction further, Kendall (1996) offered the following definitions of goals andobjectives, derived from the Accreditation Board ofthe Public Relations Society ofAm erica: A goal is "often related to one specific aspect of the mission or purpose,and commonly described as the desired outcom e of a plan of action designed to solvea specific problem over the life of a cam paign" (p. 248). An objective, on the otherhand, "addresses a specific aspect of the problem with each of several objectivescontributing toward achieving the goal" (p. 248 ). Objectives specify the com muni-cation outcome desired, the targeted public, the expected level of attainm ent, and thetime frame in which the outcom e is expected to occur.

Other scholars have offered detailed definitions of objectives as well. Forexample, Swinehart (1979) emphasized that public relations objectives shouldspecify content, target population, when the intended change should occur, whetherthe changes are unitary or multiple, and how much of an effect is desired (see alsoBroom & Dozier, 1990; Dozier & Ehling, 1992).

Research on public relations planning shows the impact that management byobjectives has had on the comm unication function (Cutlip, Center, & Broom , 1985;Dozier & Ehling , 1992; Finn, 1984; Koestler, 1977; Nager & Allen, 1984; Van-Leuven, 1989; VanLeuven, O'Keefe, & Salmon, 1988), This scholarship typicallystressed the importance of public relations goals and objectives that relate toorganizational goals. As Heitpas (1984) argued, public relations objectives shouldstem from organizational objectives, and they should be oriented toward improve-ment and clearly defined.

Few studies, though, have explored how well these ideal models for settingpublic relations goals and objectives hold up in everyday practice. One exceptionis Barlow 's (1993) analysis based on interviews with 31 corporate executives and16 academ icians, a project conducted for the Institute of Public Relations Researchand Education. He found tbat most interviewees thought goal and objective settingin public relations is actually pretty uncomplicatedm uch more so than evaluation.However, several respondents disagreed, "I don't worry so much about evalu-ation," one executive said. "I worry m ore about setting [public relations] goals that

7/28/2019 6433382

4/34

106 HONthey regard as meaningful is extremely difficult, when communications is only afactor. What hits management directly is what counts with them, and trying to fathomthat in terms of realistic goal setting poses real problems, (Barlow, 1993, p, 15)Whatever their frustrations, though, most practitioners in this study acknow-ledged their companies' process for setting public relations goals and objectiveshas become more formalized as the demand for measurable results has increased.Obviously, the systemization of public relations planning is a necessary precursorto meaningful public relations evaluation.

RESEARCH ON PUBLIC RELATIONS EVALUATIONResearch on public relations evaluation can be divided roughly into several maincategories. Of course, some studies have had multiple objectives and fall into morethan one category. However, for the purpose of explication, multipurpose studiescan be grouped here according to what seems to be their main purpose.

The following is by no means an exhaustive review of the literature relevant topublic relations evaluation. The goal of this synthesis merely is to provide someexamples of the major topics that have been explored (see also Chapman, 1982;Jacobson, 1980;M arzano, 1988, Tucker &Shortridge, 1994; Wiesendanger, 1994;Williamson, 1995).

Analyses of Communication EffectsOne category of research relevant to public relations evaluation explains howcommunication effects happen and how threats to internal and external validitylimit causal inferences (D. T. Campbell & Stanley, 1963; Flay & Cook, 1981;McGuire, 1973; Mendelsohn, 1962,1973; O'Keefe & Reid, 1990; Rogers & Storey,1987; Webb & C ampbell, 1973). In otber words, linking the cause (public relationsactivities) to some effect (generally assumed to be cognitive, attitudinal, or behav-ioral) is neither easy nor straightforward.

Yet, Reeves (1983) argued that meaningful evaluation of communication effectsgoes beyond issues of research design. Evaluation must include evidence alongseveral dimensions. That is, effective evaluation m ust answer all of the following

7/28/2019 6433382

5/34


which implies that communicated messages necessarily will cause changes inknowledge, attitude, and behavior (in that order) among target publics. Pointingout fallacies in this mode!, they detennined the likelihood of achieving behaviorchange with any member of a target public is only about 0.04%. They explainedthat, as program objectives move from message exposure to behavioral change, thelikelihood of achieving those objectives decreases dramatically.

Prescriptive ResearchThe next category entails scholarship whose main goal is to outline tbe evaluationprocess and point out the importance of evaluation for public relations. In Caro(1977), for example, various types of evaluation were described, and several casestudies were offered. Tbe au thors in this collection of readings argued strongly forthe inclusion of evaluation in all social programming.

Weiss's (1972) work also made a strong case for evaluation. However, shediscussed numerous constraints such as organizational resistance to change and thetendency of evaluation to show little or no effect. She also mentioned that practi-tioners too often attempt to evaluate programs for w hich there is no clear statementof purpose, or if objectives are concrete, they are stated as comm unication activitiesrather than program effects.

The importance of public relations evaluation tbat measures impact rather thanprocess was reiterated by Broom and Dozier (1983). They asserted that achievingspecific outcomes detailed in public relations program objectives is the yardstickby which success must be measured. For them, focusing on output or viewing publicrelations ou tcomes as unmeasurable inhibits tbe developm ent of public relations asa management function.These themes were emphasized again by Dozier (1985), who contrastedpseudopianning and pseudoevaluation with true planning and evaluation in publicrelations. He discussed the tendency to confuse communication products withdesired ends. In other words, organizational resources are devoted to measuring

Related to prescriptive research about public relations evaluation is McElreath's (1977) discussionof theoretical models of evaluation research. He posited two frameworks for understanding publicrelations evaluation.

The first of these, a contingency model, is concerned with conditions that predict whether public

7/28/2019 6433382

6/34

108 HON

output such as number of news release placements and publications rather thanspecifying desired effects and measuring whether these ends were achieved.Another example of prescriptive research was provided by O'Neill (1984), whoexplained the importance of measuring public opinion as part of public relationsevaluation. He argued that survey research on corporate reputation is vital giventhat organizations increasingly are expected to be more responsive to publics. Thus,more public relations research is needed to provide practitioners with informationon how publics perceive an organization and how perceptions change.Similarly, Stamm (1977) warned against collecting evaluation data on the wrongvariables or collecting data at the wrong time. For him, meaningful evaluation is

contingent on practitioners' situating evaluation in the proper cognitive context.Practitioners, in conjunction with clients or managem ent, must first determine thewhen, where, how, and what of evaluation if the process is lo be useful (.see alsoAtkin, 1981).Broom and Dozier's (1990) textbook. Using Research in Puhlic Relations:

Applications to Program M anasemeni, is the most comprehensive p rescription torpublic relations research and evaluation, ln this text. Broom and Dozier discussedhow research fits into public relations management, arguing that evaluation is anecessary component of sophisticated public relations practice. That is, withoutevaluation data, practitioners cannot plan and manage communication programsmost effectively.Case Studies of Public Relations EvaluationAnother category of evaluation research was exemplified by a 1977 special issueof Public Relations Review, which featured examples of public relations researchin a variety of institutional setting.s. For example. Grass (1977) discussed theevaluation of DuP ont's corporate advertising. Starting in the 1930s, DuPont begantracking changes in public attitudes toward DuPont and other companies. DuPontfound that, over time, its corporate image advertising was having the desired effect,especially its television advertising. People who had been exposed to the campaigndemonstrated an overall shift toward favorable attitudes, especially in areas ad-dressed by the campaign. Yet, despite this success. Grass concluded that DuPontwas unable to determine the bottom-line value of DuPont's significant investmentin the campaign.

J. E. Grunig (1977a) studied the effectiveness of internal communication at the

7/28/2019 6433382

7/34


J. E. Grunig's (1977a) study suggested that communication effectiveness ismaximized by directing communication resources to those publics most likely toneed and want information. He uncovered four different em ployee publics and twopublications used most frequently among them. All publics expressed a desire formore information about administration, a topic not covered in any publication.Reviewing J. E. Grunig's work, Franzen (1977) argued that Grunig's situationaltheory provides a meaningful framework for evaluating communication effective-ness.Another case study of public relations evaluation was offered by Tirone (197 7),who discussed the Bell System's attempt to develop measures that would assessthe overall performance of public relations. To m easure the efficiency of employeecommunications. Bell gauged how effectively a publication was distributed; thecompany also measured readability and awareness and understanding amongreaders. Assessing the effectiveness of media relations. Bell conducted a nationalstudy to determine the media's evaluation of the company's materials. The com-pany also content analyzed news stories. However, measuring the effectiveness ofBell Sy stem 's advertising about long distance services proved to be difficult, as didlinking the company's community and educational relations to effectiveness in

public relations. With community relations, it was hard to isolate public relations'contribution because so many people were involved (see also Tichenor, Donohue,& Olien, 1977). The problem with educational relations was a lack of clearobjectives against which to measure performance. Tirone concluded that, althoughsome component evaluation measures had been developed at Bell, a total systemfor capturing the value of public relations was elusive.In the same issue of Public Relations Review, Marker (1977) discussed howArm strong Cork Company developed a systematic program for evaluating the value

of its product publicity. He began by explaining that the success of his publicrelations efforts used to be judged solely by volume of press generated. However,when a marketing executive asked how much all the publicity was really worth.Marker w as forced to develop m ore meaningful measures. One he developed wasreturn on investment, which he calculated by taking the dollar value of print spaceand broadcast time generated and dividing it by actual project expenditures. Thedepartment also began linking custom er inquiries to specific placements. Sum mingup the success of the evaluation p rogram. Marker concluded: "N o one asks us whatPR is 'worth' any morethey know what it's worth. And they can see much moreclearly how it ties into the corporate marketing effort" (p. 59).

7/28/2019 6433382

8/34

110 HON

individual participation. They also suggested that the finding of few effects may bedue to the campaign's success at merely reinforcing existing behaviors that other-wise might have discontinued (see also Files, 1984; Hyman & Sheatsley, 1947).

A focus on research in public relations was revisited by the Public RelationsReview in a 1990 special issue. One article profiled a readership survey conductedby Pavlik, Vastyan, and Maher (1990). These researchers found moderate supportfor their hypotheses that level of organizational integration (functional and psycho-logical) and importance of organizational su rveillance (system functions and socialnetwork) among employees were related positively to level of newsletter reader-ship. This research helped evaluate who is likely to read the newsletter and whatcontent readers are looking for.

Another case study was afieldexperiment designed to test learning from a healthcampaign. Rosser, Flora, Chaffee, and Farquhar (1990) evaluated the effects of a6-year campaign designed to provide information about heart disease. Not surpris-ingly, they found that a number of variablessuch as age, risk, gender, level ofprior knowledge, and educational levelaffected participants' learning from thecampaign.

More recently Watson (1995) profiled four case studies of public relationsevaluation in the United Kingdom . He chose this methodology because he thoughtthe models of evaluation promoted in the U.S. literature "have not been testedagainst the normal constraints of practice " (p. 4) . Am ong the case studies, Watsonfound both a short-term and a long-term model of public relations evaluation. Inone short-term case, a 3-month lobbying cam paign, evaluation of effectiveness was"quickly visible" and "could be expressed as a Yes or No result" (p. 4). Thegovernment found money for a project and the organization's lobbying effortsceased.

Two of the case studiesa major industrial redevelopment and a proposal fora new communityinvolved long-term evaluation efforts. As Watson (1995)explained;The planning and developtnent issues embodied in the two projects were so sensitivethat no pre-testing of attitudes could be undertaken. In one of them, it may still be toosensitive to poll some publics. At the industrial redevelopment, research has beenused to validate the community relations programme and modify it for the future. Aniterative loop has been used and continues to be used to sustain a continuitig publicrelations process, (p. 4)

7/28/2019 6433382

9/34


data before the public relations intervention to the period after the public relationsactivity. Thus, one can infer what would have happened without the interventionas well as the effect of the intervention. Although Knobloch's study is limited topublicity tracking, sophisticated methods such as time series analysis may providea framework for modeling other public relations functions and outcomes (see alsoEhling, 1992).Some case studies of evaluation have attempted to measure relationship indica-tors rather than informational, attitudinal, and behavioral effects on publics. Mostoften, the theoretical framework used is coorientation, which posits comm unicationunderstanding, accuracy, and agreement as measures of a relationship (see Broom,1977; Broom & Dozier, 1990; Dozier & Ehling, 1992).One example comes from Bowes and Stamm (1975), who determined in theirstudy of a public agency that the agency's information sources did not increaseaccuracy among publics about the age ncy's stands even though the public believedthat the com munication did. These researchers also discovered that there was moreagreement among groups studied than the groups thought. Bowes and Stammconcluded that relationship measu res, such as accuracy and agreem ent, can providemeaningful evaluation data that migh t be missed by self-reports of knowledge andattitude (see also J .E . Grunig & Stamm , !97 3;H esse , 1976; Knodell, 1976;Stegall& Sanders, 1986).

Ferguson (1984) suggested that relationships between organizations and publicsbecome a primary focus of theory building in public relations. She posited manyvariables related to public relationships that could be evaluated: the degree to whichthe relationship is dynamic versus static and open versus closed, the degree ofsatisfaction one or both parties derives from the relationship, how much controlparties believe they have, the distribution of power in the relationship, whetherparties believe they share goals, and whether there is understanding and agreement.Despite Ferguson's (1984) call, though, little evaluation of public relations hasfocused on relationships. Making this point. Broom, Casey, and Ritchey (1997)said:

Many scholars and practitioners say that public relations is al! about building andmaintaining an organization's relationships with its publics. However, anyone read-ing the literature of thefieldwould have difficulty finding a useful definition of suchrelationships in public relations. Instead, it appears that authors assume that readersknow and agree on the meaning and measurement of the important concept ofrelationship. Unfortunately, the assumptioti is tiot supported by evidence, (p. 83)

7/28/2019 6433382

10/34

112 HON

Also exploring evaiuation of relationships in public relations is a vask forceestablished in 1996 at an Evaluation Summit sponsored by Ketchum PublicRelations and the Institute for Public R elations Research and Education, As KittyWard, president of K. Ladd Ward and Company and member of the task force,explained:My perspective is that most, if not all, of marketing and public relations has to dowith changing or building relationships. 1 wonder if we shouldn't go at this (publicrelations measurement) from the point of view of evaluating these relationships-(quoted in "Seeking Minimum Standards," 1996, p. 16-17)Another task force member, James E. Grunig, a professor at the University ofMaryland, agreed, arguing that most of the outcomes practitioners try to measuretend to occur "too fm* down the road" (if at all) to be helpful in dem onstrating publicrelations effectiveness ("Seeking Minimum Standards," 1996, p. 16). Given thisdilemma, he argued that public relations needs to evaluate maintenance strategies(e.g., building coalitions of management and publics) that are predictors of rela-tionship outcomes such as commitment and trust.

Status and Types of Public Relations EvaluationThe final category of research about public relations evaluation includes Linden-mann's 1988 survey of research activity among public relations professionals inthe United States (published in Lindenmann, 1990; see also Booth, 1986; Broom& Dozier, 1983; Chapm an, 1982; I E. Gru nig, 1977b; Leffingwell, 1975). Hisresults suggested a mixed picture. A clear majority of respondents believed thatpublic relations evaluation is and can be an integral part of programming. Morethan half agreed that it is possible to measure public relations outcomes, impact,and effectiveness precisely. Moreover, most respondents said they allocate fundsfor public relations research and that the volume of research projects they areinvolved in has increased.

On the other hand, Lindenmann (1990) found that more than 90% of surveyrespondents believed that public relations research is still more talked about thanactually done. About 70% thought that most public relatiotis research is still "casualand informal rather than scientific and precise " (p. 15). Eindenniann concluded that,although the status of evaluation in public relations had improved since the 1960s

7/28/2019 6433382

11/34


Bissland (1990) found three main categories of evaluation among the cases: (a)measures of communication output (e.g., quantity of output, number of mediacontacts, quality and quantity of media placements), (b) measures of intermediateeffect (effects on key audiences such as number who attended to the message orparticipated in an event; audience feedback; and behavior science measure-ments awareness, attitude, understanding, behavior), and (c) measures of organ-izational goal achievement (e.g., numbers of products sold, new mem bers recruited,funds raised, legislation passed or defeated).

The most frequently used evaluation measures were output and intermediateeffects36% and 39.6%, respectively. Measures of organizational achievementaccounted for only 24.6%, and some of these were inferred rather than substantiated.Yet, Bissland (1990) was able to show m eaningful progress in evaluation duringthe decade exam ined. The number of different evaiuation methods rose from a meanof 3.6 to 4.57, which was a statistically significant difference (p < .01). In1988-1989, more practitioners were using behavioral science measures44.4%,up from 25 % in 1980-1981 (significant a tp < .05). Also, the increase in m easuresof organizational goal achievement was significant28.6%, up from 18.5% (p

7/28/2019 6433382

12/34

114 HON

and em ployers' unw illingness to devote resources to evaluation, and practitioners"reluctance to tie objective performance measures to their programs.Watson (1995) later cross-checked his U.K. survey with a sample of 30 practi-tioners from 25 countries. He found that the barriers to evaluation uncovered in theU.K. survey were mirrored worldwide: lack of time, lack of personnel, lack ofbudget, cost of evaluation, doubts about usefulness, lack of knowledge, andaversion to scientific methodology (see also Bae rns 's 1993 study of evaluation inGermany).Other researchers focused specifically on identifying types of public relationsevaluation. For example, Lerbinger's (1977) analysis of corporate use of research

identified four main categories: environmental monitoring, public relations audits,comm unication audits, and social audits. Environmental monitoring was used byorganizations to gauge public opinion and observe social and political events thatmay impact the organization. The most common research technique Lerbingerfound was the public relations audit, the most comprehensive of which involve,^assessing relevant publics, an organization's standing with publics, issues ofconcern to publics, and power of publics. A communications audit evaluates theeffectiveness of an organization's communication products among internal andexternal audiences. Typically included here were readership and readability studiesand content analysis of publications. Finally, Lerbinger found that organizationswere using social audits to assess how they are performing as corporate citizens,although there was little agreement about what this evaluation method entails (seealso Reeves & Ferguson-DeThorne, 1980).

Also addressing types of evaluation, Swinehart (1979) outlined four levels otQ\a\uation: process, quality, intermediate objectives, and ultimate objectives. Proc-ess refers to measuring the preparation and dissemination of communicationmaterials. Evaluation of quality has to do with features of comm unication materialsor programstheir accuracy, clarity, production values, and suitability for theaudience. Evaluating intermediate objectives typically involves looking at precur-sors, such as media placement, to ultimate objectives. This last category ultimateobjectiveshas to do with assessing effects on the target audience's knowledge,attitudes, or behavior.

Similarly, Lindenmann (1993) presented a yardstick for measuring publicrelations effectiveness, which included basic, intermediate, and advanced evalu-ation. The basic level involves measuring outputs or media placements, impres-sions, and targeted audiences. The intermediate level, or public relations out-

7/28/2019 6433382

13/34


uncovered three types of evaluation. Scientific /m/7flcr evaluation is the most formalof the three and involves evaluating public reaction to the organization and publicrelations programs before, during, and after implementation. Seat-of-the-pantsevaluation uses informal techniques (e.g., feedback from media contacts) to gaugeprogram impact. The last, scientific dissemination, involves systematically meas-uring the distribution of messages.

Dozier (1984) then correlated types of evaluation and practitioner roles. Hedetermined that technicians are unlikely to use any type of evaluation. Managerspredominately use seat-of-the-pants evaluation to augment measurement of scien-tific impact (see also Dozier, 1985, 1990).Lesly (1986) addressed "levels of measurability" in public relations but dealt withintangible as well as tang ible effects (p . 6; see also Lesly, 1991). He argued that manyof the most valuable contributions pub lic relations makes go beyond the outcomesof specific activities that can be measured easily with num bers. In other words, someindicators of public relations effectiveness defy numerical m easurement.Lesly (1986) called the first level of evaluating intangibles semi-specific meas-urement. As an exam ple, he mentioned practitioners' assessing "the reaction to apresentation before a Senate committee , or to an officer's speech" (p. 6). No

numerical measurement is implied and yet a judgment can be made about theeffectiveness of public relations.The next level was acceptance on the basis of judgment. This occurs whenmanagement believes on faith that public relations efforts have been effective inareas such as working with minority, community, and church groups.Lesly (1986) also mentioned recognizing the value of input rom public relationspeople as an intangible evaluation measu re. Here, public relations m ight be judgedon how effective the department is at monitoring public opinion and keeping

management abreast.Last, Lesly (1986) posited prevention and guidance as key measures of publicrelations effectiveness. He argued that preventing negative outcomes, such as badpress, may be the best indicator of public relations success. Similarly, the ongoingguidance public relations provides, "like a car operating so quietly that no one isconcerned with it," is a strong testimonial to effectiveness (p. 7).Geduldig (1986), however, took issue with the argument that intangible publicrelations outcomes demonstrate value. He maintained that public relations depart-

ments must establish concrete measurement systems to justify their existence toothers:

7/28/2019 6433382

14/34

116 HON

RESEARCH QUESTIONSGeduld ig's (1986) assertion underscores the need for a better understanding of theintricacies involved in setting public relations goals and objectives and conductingmeaningful evaluation. This study explored the challenges and opportunities in-volved in these tasks among a selected group of public relations practitioners andtop executives. The following research questions (RQs) were posed:R Q l: What do practitioners and other managers consider the main goals andobjectives for public relations at their organization?RQ2: What do they think is the link between goals and objectives for public

relations and goals and objectives for their organization?RQ3: How do they evaluate public relations?

METHODOLOGYThese RQs were investigated through in-depth, nondirected interviews with 32public relations practitioners and 10 other executives. A qualitative method waschosen to provide the opportunity for respondents to discuss research topics fromtheir point of view rather than responding to structured categories. Qualitativeresearch seemed especially fitting given the multifaceted issues and perspectivesuncovered in the literature review.SamplingThe interviewees were selected following Broom and Dozier's (1990) guidelinesfor nonprobability sampling. The type of sampling used was a combination ofsnowball and dimensional. Snowballing, or asking initial interviewees to suggestother interviewees, served to expand the list of participants from the originalhandful (drawn from the researcher's professional contacts) to the final 42.

When recruiting participants, two dimensions also were considered to ensurediversity among the group: type of organization and level ot experience. A widearray of organizations throughout the United States was assembled: corporations,government, not-for-profit organizations, associations, and public relations agen-cies. In addition, although most of the practitioners (and all of the other managers)

7/28/2019 6433382

15/34


perspective given that evaluation increasingly is stressed in public relationscurricula.Interview ProcedureWith the exception of three interviews, which were held face to face, all of theinterviews were conducted over the phone. The three RQs served as grand tourquestions (Spradley, 1979, 1980), which provided the overarching framework forthe interviews. When appropriate, probes were used to elicit explanatory informa-tion and exam ples. Also, as is typical with qualitative inquiry, interviewees broughtup related issues unprompted.

After conducting the 42 interviews and an initial review of findings, closure wasdetermined to be appropriate. At this point, the emerging data seemed sufficient toprovide meaningful insight into the areas addressed by the RQs.Snow (1980) referred to this aspect of qualitative research as "informationalsufficiency" and described its three dimensions (p. 103). Taken for grantednessoccurs when very little that the respondents say surprises the researcher and theinterviews begin to seem routine. Theoretical saturation describes diminishingreturnsthe additional data add very little novel or useful information. Finally, aheightened confidence occurs when the researcher feels the "findings are faithfulto the empirical world under study and shed light on preexisting or emergentquestions and propositions" (p. 104).Data AnalysisInterview data next were transcribed, and responses were arranged under the RQthey answered. The findings then were synthesized with the dual purpose ofrevealing common themes or trends and uncovering the range of response or towhat extent practitioners' and other managers' experiences and examples diverge.

Successfully achieving this purpose is the measure of any qualitative study'smerit. As L indlof (1995) pointed out, the conventional canons of research reliabilityand validity do not fit the qualitative paradigm very well. Based on interpretiveassumptions, qualitative research recognizes "the constantly changing character ofsocial reality" (p . 238 ). Thus, "little is gained from trying to achieve reliability" orconsistency of results (p. 238).Assessing validity is also difficult because qualitative inquiry assum es the world

7/28/2019 6433382

16/34

1 18 HONLindlof s (1995) arguments, however, do not imply that qualitative researchers

lack credible and dependable data. As he said: "Basically we want to inspireconfidence in our readers (and ourselves) that we have achieved right interpreta-tions. Notice that I do not say the right interpretation. There are many possibleinterpretations of a case" (p. 238).One method for ensuring meaningful interpretations is member checks, wherebyinvestigators elicit critiques of their data analysis (Lincoln & Guba, 1985). Themember check conducted for this study involved 10 of the research participants,who were asked to review the major topics and them es that had been identified inthe data. These participants validated the efficacy of the findings by confirmingthat these topics and themes were accurate interpretations of their answers to theRQs.

FINDINGSPublic Relations Goals and ObjectivesThere was tremendous variability in respondents' explanations of public relationsgoals and ob jectives, perhaps reflecting the diversity of organizations represented.Each department divides its responsibilities differently, and not surprisingly, pri-orities tend to reflect the mission of the organization. For example, practitionersemployed by corporations tended to mention comm unication goals that support thebottom lineincreasing sales and revenue and bringing in new business. Otherpractitioners talked more about public relations' role in enhancing the image of theorganization and disseminating positive messages.

Most of the interviewees said that their public relations department does havewritten, quantifiable goals and objectives, although several acknowledged that thissystemization happened somewhat recently. Only one practitioner^a public affairsadviser for an oil companysaid that his department has no written goals andobjectives. He described a situation of great instability in w hich the business unitsof his organization are driving the goals of public affairs, while corporate publicaffairs is being downsized drastically.A few others said that their organization is moving toward a more goal-oriented

approach but it is not there yet. For exam ple, a communication manager for a tradeassociation said that setting public relations goals and objectives is a "process that

7/28/2019 6433382

17/34


reason is that the organization has suffered from a lack of data on which to basemore concrete objectives. She also mentioned people's tendency to keep "doingthings the way they did them in the past."A similar response came from a marketing comm unications manager at a privateuniversity. She said that public relations goals typically have been vague and m ore"intuitive" than explicit. She said that increased competition among universitieshas brought the realization that public relations goals have to be more precise.

The most recurring theme in practitioners' responses was that public relationsgoals and objectives have to be "strategic," meaning that public relations goals andobjectives must be tied directly to organizational goals and objectives. Many saidthat this strategic goal setting takes place every year, although some described amore long-term approach such as a 2- or 3-year plan.A comm unity liaison for a national health institute said that her departm ent setsgoals and objectives that are "in line with institutional goals and overall objectivesfor each year." Each person's personal goals have to support the communicationprograms that help the department fulfill the broader goals of the organiza-tion^such as increasing patient volume.A director of corporate communications for a government contracting firm also

stressed that public relations has to be "tied to goals of the organization." Herdepartment looks to the CEO 's published goalssuch as div ersif icatio n^and thenwrites a plan based on those goals.The importance of this link was emphasized by a CEO of a public relationsagency. She said that too often public relations goals become separate from theorganizational goals and then the function seems expendable. She warned againstcreating "isolated, separate" goals such as increased publicity that are merely ameans to an end. For her, public relations goals should match clien ts' broader goals,

such as increasing market share or sales.Several other agency executives also emphasized that public relations goals mustfocus on the "end result" expected. As one senior vice president said:The first thing you ask the client is, "What do you want to happen as a resultof PR activity?" You try to get as specific as possible . It can be a change inawareness, buying behavior, but not "supp orting" som ething. The client hasan outcome in mind. At the end of your contract, they are going to make adecision about whether you did what they want.

7/28/2019 6433382

18/34

120 HON

As companies become more and more fixated on the bottom line, maintaininga healthy cash flow, cutting the fat, on downsizing or rightsizing, it becomesmore and more important for the public relations function or communicationsfunction, like every other function, to be measurable, to show measurableresults, and that of course opens the door to really good, solid, quantifiableobjectives.Among the other managers who were interviewed, the most common responsewas that public relations' ultimate aim is communicating the image of the organi-zation. An executive vice president of a health association was emphatic on thispoint:Our success depends on our ability to communicate with the publicbeingbelievable as an organization, giving them (the public) credible information.Raising funds, being supported by the public, depends on our public image,PR is part of that image.Similarly, a president of a state university said that the goal of public relations

is "to tell the story of the institution, to inform the public about w hat's going on, tolet people know the vaiue they earn for the investment they make in the institution."He also noted: "Obviously, there is a 'best foot forward' side of PR, which I thinkis their ongoing basic responsibility."A CEO of a food products company mentioned that public relations should becarrying the organization's "proper message" to internal and external audiences. Asenior vice president for a health benefits provider also referred to public relations'responsibility as "any involvement that gets [name of company] communicated

externally and internally."Only one top executivea manager of federal relations for an oil com-panywas indefinite about public relations' mission. He said that communicationgoals are "nebulous" and "hard to pin down." Direction for public relations is influx, depending on the whim of Congress and federal and state regulatory agencies.Yet, the overall scheme is to push for legislative initiatives that do not "negativelyaffect" the organization's refining and marketing efforts.

Public Relations and Organizational Goal Achievement

7/28/2019 6433382

19/34


It's very simplefrom our magazine, to the way we market and put on oureducation program s, through our development of membership and our serv-ice to mem bershipeverything we do is promoting and enhancing respon-sible and effective philanthropy. W e do n't do it unless there is a direct link.Similarly, a director of public relations for a health benefits provider said thatpublic relations is part of "every thread of the fabric" at her insfitution"sointertwined, you can't separate" public relations goals from the organization'smission. She explained that, because the organization's activities are controversial,the agency "inherently needs PR to explain [itself] to constituents and provide achannel for constituents to give feedback."The link also was clear to a director of public affairs for a highway safetycoalition. She talked about how crucial the organization's media relations is tokeeping issues before the public and Congress. She also thought that the presidentof the organization recognizes effective public relations is the organization'slifeline. "The president truly values PR; she is media savvy ," she said. "If anything,maybe she relies on it too much in the 11th hour. But, that's a nice problem for aPR person to have."A national vice president of public affairs for a health association also had notrouble associating public relations with organizational goals:By constantly putting our messages out there and associating them with ourname, the public has a heightened sense of our mission and is more pliantwhen it comes to supporting it. Our role is that controversies are headed off,successes are featured, information flows constantly, and that, in turn, createsvery high name recognition. It's not by accident that these things happen.That's our role in making sure that the organization is successful.However, some practitioners thought the connection between public relationsand organizational goal achievement is obscure. For example, a vice president ofcorporate comm unications for a mortgage company thought that, when it comes tobringing in new business, public relations' contribution is not so obvious. "It'sdifficult to link getting loans to something we have done," she said. "It's more inthe relationship building than bringing loans in the door. It's a lag effect."A marketing specialist for a management consulting firm also alluded to thedifficulty of definitively relating some public relations outcom es to organizational

7/28/2019 6433382

20/34

122 HON

to measure. PR has not been able to provide quantitative, tangible results, butit's good at providing qualitative results.A CEO of apublic relations agency also talked about the difficulty of convincingothers of the relation between public relations goals and organizational success:Sometimes one of our biggest challenges is proving our value. The more wecan align PR goals with corporate goals, the betterto show that PR is notjust in support of those goals, but very much a part of achieving those goals.The need for pub lic relations to prove its worth also was emphasized by a publicrelations director for a private school. She too believed that public relations needsto show how its goals fit into the organization's. "You have to look at the criterionof whether or not it (a public relations goal) is contributing to the bottom line ofthe organization or its goals," she said. "If it doesn 't directly contribute, then I wouldsay it's not necessary."Perhaps the most grim situation was described by a communication manager foran animal rights and conservation organization. She described the link betweenpublic relations and the organization's mission as "not very sn-ong." She thoughtthere was a "huge gap between what public affairs does and what organizationalleaders think." Tbe communication manager said: "The ideal model works only onpaper. ... I don't see communications incorporated as much as it could beexceptin a crisis. Then it's, 'Man the torpedoes; you guys do som ething.' "This practitioner thought part of the problem is that others in the organizationdo not understand public relations because the outcomes tend to be less tangiblethan other functions. Making this point, she recalled how, after one of her particu-

larly successful efforts, a scientist at her organization grudgingly stated during ameeting: "My [new[ advice to everyone here is don't run away from (name ofpractitioner] when she comes down the hall."CEOs and other managers offered a variety of perspectives about how publicrelations goals relate to the organization. A CEO of a food products companydescribed a "firm link" between public relations and the organizational mission,noting that the vice president of corporate communications is very involved instrategic planning for the organization.However, a development director for a private university acknowledged thatpublic relations has not fed into her activities as much as it could. She said, though,

7/28/2019 6433382

21/34


organizational goals is not "as strong as it is for established organizations withspecific strategies for pu blic-corpo rate stewardship." He described his com pan y'spublic relations activities to date as reactive, stating that proactive public relationshappens only as companies evolve. "Most of the people here aren't that aware ofprofessional PR," he said. "Il's not totally respected because of the difficulty indeveloping value proofs that say, 'If you put the resources here, this is what youget back. '"Still others provided more perspectives on the connection between publicrelations and organizational goals. A president of a state university said that "PRis the window on the external world about the institution and its mission and how

well it is progressing toward realizing its mission."Similarly, the president of a government contracting firm beheved that publicrelations supports the organization's strategic plans: "Every company has a visionof where it wants to be. If we want to change the image, public relations wouldassist in providing that support." However, he said that there is not "much connec-tivity on a day-to-day basis" between public relations goals and the organization'smission. Instead, public relations' contribution is more long-term.

Public Relations EvaluationResponses about public relations evaluation revealed a continuum from virtuallyno evaluation to formal and ongoing efforts. However, despite this range, none ofthe interviewees disregarded the need for public relations evaluation. Some practi-tioners said that, although they do not feel pressure to show how the needle movedfor every public relations activity, they still must demonstrate "valu e." Others wereemphatic about showing m easurable results.Not surprisingly , many responden ts comm ented on how difficult public relationsevaluation is. They bemoaned inadequate resourcestime, staff, and moneyaswell as the intangible nature of some public relations goals.

A director of university relations for a state university evoked the well-wornphrase that "evaiuation is the 'Achilles hee l' of public relations." As he elaborated:It' s real easy to measure things that do n't matter and very difficult to measurethings that do matter. The end result of our efforts here should be changes,enhancements in attitudes of carefully defined publics. ... It doesn't matter

7/28/2019 6433382

22/34

124 HON

mentioned, though, that public relations' needs are considered "perimeter" ratherthan "center" to the study.A vice president of membership for a trade association was skeptical in generalabout public relation.s evaluation. She believed that effectiveness is just too slip-pery:I'm not sure you ever really know with public relations. Yes, you can evaluatean event, a mood, an opinion poll, but from the view of 20-odd years, thatcan change overnight. ... Any company that says their public relations isabsolutely effective is kidding itself.

A similar theme was heard from a public relations director at a trade association:Effectiveness in communications is always very difficult to measure. Youspend an awful lot of energy and time and resources and you probably do n'teven know the effect until maybe years lat er .... You just work on going afteryour goal. And, i t's not that you are hit or miss; you know certain things fromexperience. But, you do n't sit down and analyze that.A vice president of corporate communications for a food products company alsowas doubtful about public relations evaluation: "We are not turning in sales reportson aweekly or monthly basis by which we can be judged. A lot of this is intangible."He acknowledged, though, that "the demand for measurable results is increas-

ing " and that research is "vita l" and often "the bread and butter of many successfulprograms." However, at his company, the budget is just not there. As he explained:

The difficulty really comes in the budgetary area. When you are having towatch costs very closely, there is litUe support for going outside to embarkupon a major and costly program to help you determine just how well youare doing.Despite constraints, though, most practitioners were doing some sort of evalu-ation. The measures used typically varied by type of program and target audience.Several practitioners referred to their evaluation as informal, m entionitig feed-

back from audiences or others in the organization about public relations' productsor services. For example, a publications editor for a trade association said that her

7/28/2019 6433382

23/34


campaign. "The days of companies spending lots to get their name in the paper areover," she said. "Our clients are specific."Other agency executives disagreed. A comm unication associate for a manage-ment consulting firm thought that clients often want nothing m ore fi-om publicrelations than "get my name in the paper."A CEO of another agency thought selling clients on evaluation can be difficultbecause clients think that public relations is free advertising and therefore shouldnot cost a lot. Still another agency vice president thought clients often do notunderstand research or place little faith in the measure of public relations activities.Thus, evaluation at her agency amounts to little more than tallying media impres-

sions or counting heads at special events.Some practitioners were doing little or no evaluation but planned to do more. Adirector of public relations for a health benefits provider explained that, before shejoined her company, the CEO's yardstick for evaluating public relations was"whether or not a good time was had by all," She wants, however, to beginconducting survey research that would provide benchmark data about levels ofbrand awareness. The survey then would be repeated every 6 months or each year.She believes other managers will support the project because "they would like toknow they have measurable results; TQM [Total Quality M anagemen t] has broughtthis m entality."

Evaluation techniques described by other practitioners were those one wouldexpect to find in public relations: conducting focus groups and readership andcustomer surveys; monitoring government research; and tracking media placement,attendance at events, store traffic, calls to toll-free numbers, and "hits" on theorganization's World W ide Web hom epage.Some practitioners evaluated behavioral objectives almost exclusively, which

they feel m akes measurem ent straightforward. An exam ple was a director of alumnirelations for a state university who said that evaiuation is "kind of easy." Herdepartment gauges its ultimate success by tracking donations.Similarly, an executive director of a trade association explained that at hisorganization "measurement is built in," and the meastjre is "specific regulatoryaction." Referring to the organiza tion's mem bers, he said: "They d on 't care if weincreased awareness or opinions if their ultimate goal was not achieved. They do n 'tcare abo ut winning the battles if you lose the war. They do n't w ant to spend moneyon the ba ttles."A management associate for a passenger railroad also thought that the acid test

7/28/2019 6433382

24/34

126 HON

Perhaps the most debatable issue among these practitioners had to do withevaluating media placement. Almost all of the interviewees mentioned usingclipping services. However, opinions about the value of doing so varied greatly.A senior vice president of a national agency summarily dismissed clippings:

"You might as well count press releases. ... You still have to tie it (placement) tosome impact."However, a director of corporate communications for a governm ent contractingfirm said that her organization has "found very elaborate ways to quantify howmuch press we have gotten and what kind." She said that the company has becomemore "sophisticated in quantifying that (media placement) in terms of advertisingdollars." She also is tracking "inroads into trade publications and newspapersimportant to us."

Most other practitioners were ambivalent about the worth of evaluating mediaplacement. They acknow ledged that placement may indicate message exposure. Asone agency C EO said, placement is not the "panacea'* of public relations evaluation .One example of systematic, ongoing evaluation was provided by a director ofexternal relations for an international exchange organization. She explained that,before the association's yearly conference, her department does a "needs assess-

ment survey." The results are used to plan the conference. A follow-up survey andtocus groups then are conducted . "We tet\d to evaluate projects like this," she said."The board and members are open to evaluation, research, getting feedback frommembers, putting it into action."Only one practitionera communication manager for a media and technologycompanyexplicitly pointed out the connection between public relations evalu-ation and organizational goal achievement:

One way that we measure [public relations goals and objectives] is thatperformance plans are supposed to be linked back to goals for the organiza-tion. We haven't done a good job of communicating the vision of thecompany, so people have a hard time seeing the link.When the CEOs and other managers were asked about measures of effective-ness in public relations, their answers were similar to those provided by thepractitioners successful media placement and desired changes in attitudes andbehaviors. Moreover, every single top executive stressed that effective publicrelations successfully builds the organization's image.

7/28/2019 6433382

25/34


that required experts in our area, we were contacted. We were known. Whena significant issue arose that could have been a PR disaster, it was diffusedand diffused rapidly. The marketing materials occurred in such a way thatnothing w as disruptive; the proper message was out; it was timely. Supportingmaterials had the look, the feel; there was synergy.Similarly, a president of a state university said there is "one way" to measurethe effectiveness of public relations: "the success of the organization in havingstories about the organization in the media, the frequency and quality of commu-nication with the public, the attitude of alumni, parents, and students about the

institution." He also mentioned growth in private giving as an organizational goalrelated to successful public relations.Some of the executives also related public relations evaluation to performanceindicators for the organization. A market analyst for a passenger railroad was veryclear about the ultimate measure of effective communication: "In terms of ourperformance, [we ask], 'Do our revenues go up?' If we are really a good, competi-tive form of transportation and that word is out, then we should be able to make alittle more money."When probed about whether others in the organization understand the linkbetween effective public relations and increased revenue, he said that the connectionhas become clear only within the last 6 months: "Before that, it (public relations)was pretty much isolated, and when it came time to make cuts, it was the first. Therewasn't the understanding."The company's new-found appreciation for public relations came from sheer"desperation," he said. "What we had done before just wasn't working." He saidthat a new management team with a new philosophy then was recruited: "These

new individuals understand communication betterinternally and externally."Other executives recognized the connection between measures of effectivepublic relations and organizational goal achievement but acknowledged that thelink is not always explicit. For example, a CEO of a national health institute saidthat public relations at his organization is evaluated through physician and con-sumer attitude surveys. Yet, he said the line is "fuzzy" between attitude data andperformance indicators such as "success with the legislature, success with gettingnational grants, [and] numbers of national news pieces."A senior associate at an animal rights and conservation organizafion echoedthis theme. She explained that, at her agency, evaluating a direct marketing

7/28/2019 6433382

26/34

128 HON

This notion of accepting public relations on faith seemed to run through mostof the CEOs' and other managers' responses."^ An executive vice president of anational heaith association clearly believed that the awareness and trust publicrelations creates are directly related to his organization's ultimate measure ofsuccessreducing disease and risk factors for disease.He made this point with an example about women and breast cancer:Getting women to change their behavior is in part dependent upon our publicrelations image because when we go to the public, there are other messagesthat they are getting. Our image has to be consistent, good, [and] accurate sowomen will believe us and get mammograms. The public relations part ofthat has to do with credibility.

DISCUSSIONThis example illustrates the main point made in much of the literature about publicrelations evaluation: Measuring the outcomes of public relations programs providesdata needed to demonstrate that public relations helps organizations and clientsmeet their performance goals.How ever, as this study underscores, setting goals and objectives and conductingmeaningful evaluation in public relations continue to be complicated tasks. Part ofthe problem is that public relations itself encompasses so many different activitiesthat goals and objectives will vary, depending on organizational priorities. It goeswithout saying then that no single evaluation formula will be appropriate for allprograms and organizations.

Yet, despite the variable responses among these practitioners and CE Os, severaltrends are discernible. The first is that these practitioners seem to be moving towardmore systematic public relations programming. All but one practitioner said thathis or her organization has written goals and objectives for public relations. It isperhaps not surprising that the one organization lacking written goals and objectivesis downsizing its public relations department.

Another striking theme is the long-range plans several interviewees had forbuilding more formal evaluation into their public rclations efforts. Most often, thejunio r practitioners were the ones who described such schemes. So, although someof these practitioners may be overly optimistic,'' their outlook suggests that evalu-ation may be becoming a more integral part of public relations programming.

7/28/2019 6433382

27/34


A related point is that several practitioners seemed to be educating others in theorganization about measurement issues in public relations. An example is thepractitioner whose CEO used to think that "a good time was had by all" is theultimate gauge of public relations effectiveness.

Perhaps the most encouraging finding is the understanding among these practi-tioners and CEOs of how public relations feeds into the strategic goals of theirorganization. Although most of the time this connection was inferred only, obvi-ously m ost interviewees do not see public relations as an isolated function. Instead,they can articulate clearly how the ultimate success of the organization is inextri-cably linked to effective com munication. As one practitioner said, public relationsis part of "every thread of the fabric" at her organization. Similarly, the presidentof a state university thought that public relations is no less than the university's"windo w" to the external world about the university's mission and progress towardthat mission.Of course, some exceptions were revealed. Several interviewees describedsituations in which public relations continues to be devalued. One reason given wasthat others in the organization do not understand the function. Making this point,one of the senior executives added at the end of his interview that professional

associations in public relations need to make sure business students are exposed topublic relations courses in college. He mentioned that, in his own marketingcurriculum, public relations was only a "very small piece."He went on to describe what he sees as a real research opportunity for publicrelations practitioners:We are becoming more and more of an information society. Public relations[practitioners] know where information is, where it can be obtained. Theycan becom e brokers of information for organizations very easily. [There willbe] more and more data and power to crunch. Proactive public relationspeople of the future are going to anticipate issues and convince organizationsto position themselves and have the right numbers [to do so].Other interviewees expressed frustration that the effects of public relations arehard to isolate and too long-term to connect back to practitioners' efforts. As oneexecutive mentioned, coming up with "value-proofs" for public relations programs

is very difficult. Another practitioner argued that, even if you could, people'sperceptions can change overnight.

7/28/2019 6433382

28/34

130 HONdistressing that not one practitioner mentioned turning to public relations academi-cians for research help. Obviously, more collaboration between practitioners andacademicians would be mutually bet\eficial.^

Despite constraints, though, almost ail of the practitioners are doing some sortof evaluation. They mentioned virtually all of the different types of evaluation thathad been identified in earlier research.Many of these practitioners still rely heavily on informal methods such asgathering feedback from audiences. In addition, several feel that assessing mediaplacement remains a valuable indication of public relations success. A couple ofthe CEOs certainly seem fixated on media exposure.Other interviewees thought this approach is anachronistic. These practitionersare using or moving toward more sophisticated evaluation methods. Severaldescribed plans for gathering benchmark survey data that they hope is the first steptoward systematic, ongoing research.Only two practitioners described evaluation scenarios that seem pretty close toideal. One mentioned that, at her organization, public relations research is con-ducted both to plan and evaluate. The information gleaned from evaluation then isfed back into planning. Another practitioner said that, because clients demand

accountability, formal research is built into every one of the agency's programs.However, not even these two practitioners described measuring any relationshipindicators. Another point is that all of the evaluation schemes mentioned bypractitioners and top executives seem characteristic of an asymmetrical model ofpublic relations (see J. E. Grunig & Hunt, 1984). No one offered any examples ofmeasuring managem ent's position or monitoring the organization's responsivenessto publics. Although some mentioned getting "feedback," doing so involvedgleaning information from publics alone.Finally, no one described experimental research designs whereby a strong casecould be made for causal inferences between public relations activities and specificoutcomes. Most often the link just was assumed.In many organizations, though, this understanding may be what is needed.Almost all the intervieweeseither implicitly or explicitlystressed that effec-tiveness in public relations has more to do with demonstrating vaiue than docu-menting communication effects per se. Fortunately, demonstrating value may beeasier to do (L. A. Grunig, Dozier, & G runig, 1994; Hon, 1997). Goals, objectives,

and evaluation can provide part of the answer. The task that lies ahead is findingways to capture the rest.

7/28/2019 6433382

29/34


ACKNOWLEDGMENTThis article was a paper presented to the Public Relations Division , Association forEducation in Journalism and Mass Com munication, C hicago, July 1997.

REFERENCESAtkin, C- K. (1981). Mass m edia information campaign effectiveness. In R. Rice & W. Paisley (Eds.),

Public communication campaigns (pp. 265-279 ). Beverly Hills, CA: Sage.Baerns, B. (1993, May). Understanding the development of public relations in G erman. East and West.

Paper presented at the Second European Seminar for Teachers, Practitioners and Researchers,Prague, Germany.Barlow, W, (! 993). Establishing public relations objectives and assessing public relations results (Paper

prepared for the Institute for Public Relations Research and Education). Princeton, NJ: ResearchStrategies Corporation.Bissland, J. H. (1990). Accountability gap: Evaluation practices show improvement. Public R elationsReview. 76(2), 25-35.

Booth, A. L. (1986). Strength in numbers. Public Relations Journal. 42(9). 26-30, 43,Bowes, J. E., & Stamm , K, R. (1975). Evaluating comm unication with public agencies. Public Relations

Review, I, 23-37.Broom, G. M. (1977). Coorientational measurement of public issues. Public Relations Review, 3(4),110-112.Broom, G. M., Casey, S-. & Ritchey, J. (1997). Toward a concept and theory of organization-public

relationships. Journal of Public Relations Research, 9, 83-98.Broom, G- M .,& Center, A, H, (1983). Evalualion research. Public Relations Q uarterly, 2S(3), 2- 3,Broom, G. M., & Dozier, D. M, (1983). An overview; Evaluation research in public relations. Public

Relations Quarterly. 28(3), 5-8.Broom, G, M., & Dozier, D, M. (1990). Utinfi research in public relations: Applications to program

management. Englewood Cliffs, NJ: Prentice-Hall.Campbell, C. (1993). Does public relations affect the bottom line: Study shows CEOs think so. PublicRelations Journal, 49(10), 14-17.Campbell, D. T., & Stanley, J. C. (1963), Experimental and quasi-experimental designs for re.tearch.Chicago; Rand McNally.

Caro, F. (Ed,). (1977). Readings in eva luation research (2nd ed.). New York; Ru.sse!l Sage Foundation,Chapman, R. (1982). Measurement; Alive and well. Public Relations Journal, 38(5), 28-29,Cline, C. (1984). Evaluation and measurement in public relations and organization communication: A

literature review . San Francisco; International Association ofBusinessC omm unicatorsFoun dation,Cutlip, S. M., Center, A. H., & Broom, G. M. (1985), Effective public relations (6th ed.). EnglewoodCliffs, NJ; Prentice-Hall.

Dozier, D. M. (1984), Program evaluation and roles of practitioners. Public Relations Review. 10(2).13-21.Dozier, D. M. (1985). Planning and evaluation in public relations practice. Puhlic Relations Review.

7/28/2019 6433382

30/34

132 HON

Bhling. W. P, (1992). Estimating the value of public relations and comm unication to an organi/,alion.In J, Giiinig (Ed,), Excellence in public relations and communicatiam management (pp 6 i 7-63S).Hiilsdale, NJ: Lawrence Brlhaum A,ssociates, Inc.

Ferguson, M. A. (1984, August). Building theory in public relations: Interorganizatumai relationshipsas a public relations paradigm . Paper presenled to the Public Relations Division. Association forEducation in Journalism and Mass Communication, Gainesville, FL.

Files, J. A (1984). Study shows that public information campaigns work. Public Relaiitms Inurnal40(\)^9

Hinn, D, (1984). The elements of sound public reialions planning. New York: Forum.Flay, B. R.. & Cook, T. D, (1981). Bvaluation of mass media prevention campaigns, ln R. B. Rice A

W. Paisley (Eds.). Public communication campaigns (pp. 239-264), Beverly Hills, CA: SageFranzen, R. S. (1977). An NRS internal communications study: A comment. Public Relation-; Review3(4). 83-88.Geduldig, A. (1986). Measure for measure, Puhlic Relations Journal, 42(4). 6-7.Grass, R, C. (1977), Measuring the effects of corporate advenising, Puhlic Relation'; Review f(4)

39-50,Grunig, J, E. (1977a). Evaluating employee communication in a rcsearch operation. Public Relations

Review, .?(4), 61 -6 2.Grunig, J. B. (1977b), M easurement in public relations: An overview. Puhlic Relalitms Review {(4)

5-10.Grunig. J. E. (1983). Basic research provides knowledge that makes evaluation possible. Public

Relations Quarterly, 2^(3), 28-32,Grunig, J. E,, Dozier, D. M,. Ehling, W, P,, Grunig. L, A., Repper, F. C . & White, J. (1992). Excellencein puhlic relations and communication nuiiiagemenl. Hillsdale, NJ: Lawrence Eribaum AssociatesInc,

Grunig, J, B,, Grunig, L, A,. Dozier, D, M,, Bhling, W, P,, Kepper, F, C , & White, J. (1991, June), l^ewresults confirm value of communication and the components of excellent public relations. Paperpresenled at the meeting of the Intemational Association of Business Comm unicators, WashingtonDC.

Grunig, J, B,, & Hickson. R, H. (1976). An evaluation of academic research in public relations. Publi(Relations Review. 2(1), 31-43.Grunig, J, E,, & H unt, T, (1984), Managing puhlic relations. New York: Holf, Rinehart & Winston.Grunig, J, B,, & Stamm, K, R, (1973, March/Ap ril), Comm unication and coorientation of collectivities.

American Behavioral Scientist, pp . 567-591.Grunig, L. A,, Dozier, D. M., & Grunig, J. B. (1994), lABC excellence in public relation and

communication m anagement: Phase 2. Qualitative study. San Francisco; Internalional Associationof Busine.ss Communicators Research Foundation.Hause, D. (1993), Measuring the impact of public relations: Eleclronic techniques improve campaignevaluation. Public Relations Journal, 49(2), 14-21,

Hcitpa.s, Q. (1984). Planning- In B, Cantor (Bd,). Experts in action: Inside public relaltons (pp29 1- 30 1) , New York: Longman.Hesse, M. B, (1976), A coorientation study of Wisconsin state senators. Journalism Quarterly. 5.i626-633, 660.

7/28/2019 6433382

31/34

DEMONSTRATING EFFECTIVENESS .N PR 1 3 3

Johnson, B,(1994). Prove public relations affects the bottom line. PHb/;cRe/riofi.(/mu/, 50(4), 31,40.

Kendall, R. (1996). Puhlic relations campaigns strategies: Planning for implementation. New York:HarperCollins.

Kirban, L. (1983). Showing what we do makes a difference. PKMC/;e/iaHV)/i.v QwaMer/y, 28(3), 22-27.Knobloch, S. (1996, June). What would have happened without puhlic relations? Time series analysis

a.9a(ow/

7/28/2019 6433382

32/34

134 HON

O'Neill, H. W, (1984) , How opinion surveys can he lp publ ic re la t ions s t ra tegy. Public Relation.^ Review/ 0 ( 2 ) . 3 - 1 2 .

Pavl ik, J . , Vastyan, J . , & Maher , M, F. (1990) , Using readership resc :u-ch to s tudy employee views.Public Relations Review. 16 (2), 5 0 - 6 0 .

Pr i tchi t t , J . (1994) . Public relations evaluation: Professional accounlability (Gold Pape r No . I I )Geneva, Sw itzerland: International Puhlic Relations Association.

Reeves, B. (1983), Now you see them, now you don't; Demonstrating effects of communicationprograms. Public Relations Quarterly, 28(3). 17-21, 27.

Reeves, B., & Ferguson-DeThorne. M. A, (1980). Measuring the effect of messages about socialresponsibility. Puhlic Relations Review , 6(3), 40-55 .

Rogers. B. M., & Storey, J. D. (1987), Communication campaigns, InC, Berger& S, Chaffee (Eds,).Handhook of com munication science (pp, 871886), Newbury Park. CA: Sage.

Kosser, C , Flora, J. A,, Chaffee, S. H., & Farquhar, J. W. (1990). Using research lo predict learningfrom a PR cam paign, Puhlic Relations Review, 16(2), b\-ll.

Scriven, M, (1967), The methodology of evaluation. In R. W, Typer, R, M, Gagne , & M . Scriven (Eds ),Perspective.^ of curriculum evaluation (pp. 39-83). Chicago: Rand McNally,

Seeking m inimum standards for measuring publicrelationseffectiveness, (1996 . November). [Summaryof evaluation summitl, (Available from Ketchum Public Relations Research and MeasuremeniDepartment, 292 Madison Avenue, 9th Floor. New York, NY 10017)Setting benchmarks leads to effective programs. (1993). Puhlic Relations Journal, 49(2). 16.

Snow, D. A. (1980), The disengagement process: A neglecled problem in participant-ob,servationre.search. Qualitative Sociology, 3. 100-122.Spradley, J. P. (1979). The elhnographicinterview. New York; Holt, Rinehart & Winston.Spradley, J. P, (1980). Participant ohservation. New York: Holt, Rinehart & Winston,

Stam m ,K ,R , (1977 ). Stra tegie s for eva luat ing public relat ions. PHWirftWt/(/Vw..?ffev(eH-, 5(4 ), 120-128.Stegail, S, K,, & Sanders, K. (1986). C oorientation of PRpractitioners and news personnel in education.

Journalism Q uarterly. 63. 341-347.Strenski. J. B. (1981). New co ncerns for public reialions measurement. Public R elations Journal. 37{5).

16-17.Suchman, B. A. (1967). Evaluation research. New York; Russell Sage Foundation.Sutton, S, (1988, March 2 9-A pril 3). Are we measuring Ihe rights things? PR Week, p, 9.Swinehart, J. W. (1979 ), Evaluaiing public relations. Public Relations Journal. 35(1). 13-16.Tichenor, P. J., Donohu e, G, A,. & Oiien, C. N. (1977). Community research and evaluating com muniiy

relations. Public Reialions Review. 3(4). 96-109,Tir one .J.F . (1977), Measuring the Bell Sys tem 's public relations./'Mi'//cfie/anVw5fiev/eH,.f(4), 21 -3 8.Tucker, K ., & Shortridge, R, (1994). Behavioral principles in public relations. Puhlic Relations Journal,

50(8), 56-57.VanLeuven, J. K. (1989), Theoretical models for public relations campa igns. In C. Botan & V, Hazleton

(Eds,), Communication theory inpublic relations. Hillsdale. NJ: Lawrence Eribaum A ssociates, Inc.VanLeuven, J, K., O'K eefe. G., & Salmon, C, (1988, July), Effects-hased planning for public relations

campaigns. Paper presenled to the Public Relations Division, Association for Education in Journal-ism and Mass Communication, Portland, OR.Waison, T, (1994), Puhlic relations evaluation: Nationwide sun

7/28/2019 6433382

33/34


Weiss, C, H, (1972), Evaluation research: Methods of a ssessing prof^ram effectivene.ss. EnglewoodCliffs, NJ; Prentice-Hall,

Wiesendanger, B. (1994), A research roundup. Puhlic Relations Journal. 50(5), 23-27,Williamson, D, A, (1995, October 16), Inching toward web measurement. Advertising Age, p. 48,W inokur, D,, & Kinkead , R, W. (1993), How public relations fits into corporate strategy; CE Os assess

the importance of PR today and in the future. Public Relations Journal, 49(5), 16-23,Wright, D, K. (1979), Some ways to measure public relafions, Puhlic Relations Journal, 35(1), 17-18,Wylie, F, W,, & Slovacek, S. P, (1984). Public relations evaluation: Myth, option, or necessity? pMfc/if

Reialions Review, 70(2), 22-27 .

7/28/2019 6433382

34/34

6433382

Documents