recall bias and reliability of survey data-moose hunting

7/30/2019 Recall Bias and Reliability of Survey Data-Moose Hunting

1/10

Recall Bias and Reliability of Survey Data: Moose Hunting in MaineAuthor(s): Stuart M. Mazurkiewicz, Kevin J. Boyle, Mario F. Teisl, Karen I. Morris, Alan G.ClarkReviewed work(s):Source: Wildlife Society Bulletin, Vol. 24, No. 1 (Spring, 1996), pp. 140-148Published by: Allen PressStable URL: http://www.jstor.org/stable/3782847 .

Accessed: 17/01/2012 16:21

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

Allen Press is collaborating with JSTOR to digitize, preserve and extend access to Wildlife Society Bulletin.

http://www.jstor.org
http://www.jstor.org/action/showPublisher?publisherCode=acghttp://www.jstor.org/stable/3782847?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/3782847?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=acg


2/10

140 RECALLIAS

R e c a l l b i a s a n d reliability o f s u r v e y d a t a :m o o s e hunting i n M a in e

Stuart M. Mazurkiewicz;, Kevin J. Boyle, Mario F. Teisl, Karen I. Morris,and Alan G. ClarkAbstract Moose huntingdata were collected for the 1989 Maine moose (Alces alces) huntto in-vestigate recall bias in factual data and reliabilityof hunters'opinions and preferences.We surveyedthe 900 residentmoose hunterswho participated n the October 1989 hunt.Hunterswere randomlyassigned to three groups: one group (n=150) was surveyed im-

mediatelyafter he hunt and thenagain4 monthslater; he second group(n=600)was sur-veyed immediatelyafter he huntonly;the thirdgroup(n=l50) was surveyed4 monthsaf-ter the huntonly. A total of 804 hunters 89%)responded o the survey. The 4-monthre-call for moose huntersdid not result in recall bias, which is contrary o the findingsofgeneralstudies of huntingwith annual recall. Statisticalresults ndicatedthatthe opinionand preferencedata were reliable.Keywords Alces alces, huntersurveys,Maine, recall bias, reliability

Wildlife managers often depend on surveys to pro-file characteristics and solicit opinions of users ofwildlife resources. Despite widespread use of surveydata by wildlife and other resource managers, thereare concerns about their accuracy.User characteristics (e.g., age, sex, and education)and participation in wildlife-related activities (e.g.,days and hours spent hunting) are factual data thatprofile users and their levels of use. Factual data aresubject to recall bias, which is not likely to occurwhen survey participants report their characteristics(e.g., age, education, etc.), but can be a problemwhen wildlife users are asked to recall the extent oftheir participation in wildlife-related activities. A re-cent study investigated recall bias in the National Sur-vey of Fishing, Hunting, and Wildlife-AssociatedRecreation (National Survey hereafter; Westat, Inc.1989). Data on fishing, hunting, and wildlife-associ-ated recreation during 1985 were collected in early1986 (U.S. Fish and Wildl. Serv. and U.S. Bur. of Cen-

sus 1988). Westat found that annual recall results inoverreporting participation rates relative to re-sponses based on semiannual, quarterly, and monthlyrecall periods (Chu et al. 1992). This finding wasconfirmed in a study (U.S. Fish and Wildl. Serv. andU.S. Bur. of Census, 1993, L. S. Cahoon, C. A. Riker,and T. F. Moors, Recall bias in the national survey offishing, hunting, and wildlife associated recreation,unpubl. data) conducted with the 1991 National Sur-vey.Westat, Inc. (1989) found overreporting was mostprofound among anglers due to the large number offishing trips some anglers take over an extended pe-riod, resulting in anglers including effort prior to the1-year recall period. Gems et al. (1982) comparedfishing trip data collected every 2 weeks with data fora 2-month recall period. The 2-month recall periodproduced lower estimates than data collected every 2weeks, providing further evidence of recall bias. Re-call bias arises from problems in survey instrument

Address for Stuart M. Mazurkiewicz (deceased) and KevinJ. Boyle at the time of research: Department of Resource Economics andPolicy, University of Maine, Orono, ME04469-5782, USA. Address for Mario F.Teisl: Departmentof Agriculturaland Resource Eco-nomics, University of Maryland, College Park, MD, USA. Address for Karen I. Morris: Maine Department of Inland Fisheries andWildlife, Bangor, ME 04401, USA. Address for Alan G. Clark: Maine Department of Inland Fisheries and Wildlife, Augusta, ME04333, USA.

Wildlife Society Bulletin 1996, 24(1 ):140-148 Peerrefereed


3/10

Recall bias * Mazurkiewicz et al. 141design, poor recall by survey respondents, or a com-bination of these problems.Opinion and preference dataare subjective evalua-tions. Foropinions, respondents evaluate observablephenomena (eg., crowding by other hunters). Forpreference data, respondents choose alternativewildlife-management options (e.g., changes in thehunting season). Opinion and preference datado notrecord factual information and, as such, are not usu-ally subject to recall bias; these data can change overtime. For example, if a hunter finds a new job thatconflicts with the timing of a hunting season, his orher preference for when a hunting season should beheld may differ from before obtaining the new job.However, if conditions that affect individual hunterpreferences have not changed but individual hunters'responses to an opinion or preference question varydepending on the timing of the survey, the datawould be deemed unreliable. Reliabilityfocuses onthe stability of individual responses to survey ques-tions on an initial test and subsequent retest. Al-though the reliabilityof data have been investigatedin other contexts (e.g., educational testing), ourstudy is the first to investigate reliability of wildlifeusers' opinions and preferences (Allen and Yen 1979,Kealy et al. 1988, Loomis 1989). Unreliable surveydata may be due to unstable preferences of wildlifeusers, respondents inability to answer survey ques-tions, or survey design flaws.The basic insight from the Westat, Inc. (1989)study is that recall bias in survey data for wildlife ac-tivities increases as the recall period is lengthenedand the frequency of participation in an activity in-creases. If fishing is one extreme, then moose (Alcesalces) hunting in Maine is near the other end of thecontinuum. Fewer than 1 of every 80 individualswho apply receive a moose hunting permit, and per-mit holders must wait 2 years after being drawn be-fore they can apply again. Most moose hunters make1 trip to their assigned hunting zones during the an-nual 6-dayseason. These factors reduce the recallpe-riod and limit the frequency of participation. The re-search reported here uses moose hunting data col-lected after the 1989 hunt to investigate potentialrecall bias in factual data and the reliability ofhunters' opinions and preferences.

MethodsThe surveySince 1982 Maine has conducted an annual moosehunt. Permits to hunt are issued via a lottery to 900Maine residents and 100 nonresidents. The 1989hunt was held from 17-22 October. The legislation

establishing the moose hunt requires all hunters tocomplete a survey after their hunts (Maine PublicLaw, Chapter 118, 1981). The experiment was onlyapplied to resident moose hunters.The survey was designed and administered usingthe Total Design Method (Dillman 1978), which in-cludes the design and layout of the survey question-naire in a booklet form. We used this methodologyto minimize design flaws that might lead to false con-clusions of recall bias or unreliable data. The surveywas pretested in 1988 and expanded for the 1989season.

Hunters were notified of the survey prior to themoose hunt in a packet of moose hunting informa-tion provided with hunting permits. The surveyprocess included 4 mailings: the survey, a reminderand thank you card sent 1 week later to all hunters,the same survey to non-respondents 2 weeks afterthe initial mailing, and the survey again to nonre-spondents via certified mail about 4 weeks after theinitial survey mailing.Experimental designThe test for recall bias and reliability included 3 ex-perimental groups. Respondents in group A (n =150) received a survey immediately after the hunt(Al) and were asked to complete the survey a secondtime 4 months later (A2). Respondents in group Awere not told of the retest when they initially re-sponded to the survey. This double application ofthe survey is called test-retest. Groups Bi (n = 600)and C2 (n = 150) were control groups; Group B1only received a survey immediately after the hunt andGroup C2 only received a survey 4 months after thehunt. Group B1 was a control for nonrandom assign-ment to experimental groups. Group C2 was a con-trol group that allowed for identifying potential learn-ing or anchoring effects, but it is not possible to dis-tinguish between these effects.

The purpose of the within- and across-subject de-sign is relatively straight forward. The ideal design isthe within subject test-retest of Group A because wewanted to investigate the temporal stability of indi-vidual hunters' responses to survey questions. How-ever, observations of Group A at time 2 can be conta-minated by these individuals' initial responses at time1 due to learning or anchoring effects. The reasonfor including control groups differs slightly betweenthe tests of recall bias and reliability. In terms of re-call bias, Group A may recall factual information dif-ferently in the second administration of the surveybecause they had already completed the survey once.Group C2 did not have this initial priming and onlyanswered the survey at time 2. Statistical compar-


4/10

142 Wildlife Society Bulletin 1996, 24(1):140-148isons of Groups Al with A2 and Groups Bl with C2allowed us to determine conditions where Group Agave a false reading regarding recall bias (i.e., samplestatistics for Groups Al and A2 do not differ and sam-ple statistics for Groups Bl and C2 differ). The samepattern of statistical findings can result in false read-ings regarding reliability, but for a different reason. Ifthe interval between survey applications is short, re-spondents in Group A may recall their initial re-sponses to the survey and reiterate these answers tothe opinion and preference questions in the secondsurvey application. This is an anchoring problem.Thus, the test-retest application allowed us to trackindividual respondents, while the control groupsidentified undesirable test-retest effects. The Wes-tat, Inc. (1989) study was only conducted with anacross-subject design, equivalent to our controlgroups. Thus, the Westat statistical finding are not asrobust as results generated using within- and across-subject treatments.

Although subtle, the difference between recall biasand reliability is important. We investigated recallbias to identify whether individuals can rememberfactual information. Reliability investigates the stabil-ity of opinions and preferences over time. The sur-vey interval should be long enough that respondentscan not recall (reiterate) their initial responses to anopinion and preference question, but short enoughthat factors influencing opinions and preferences re-main unchanged. Thus, although the experimentaldesigns are identical, recall bias investigates surveyrespondent's memories and reliability investigatesthe stability of their opinions and preferences.Tests for random assignment to experimentalgroups (hunter characteristics), recall bias (hunterparticipation), and reliability (opinions and prefer-ences) are based on the hypothesis:

H:O =A, =0^2 =O, 2 (1)where the overbars denote population parameters(e.g., proportions and means). Failure to reject thenull hypotheses for hunter characteristics supportsrandom assignment to experimental groups, whereasfailure to reject the null hypotheses for participation(opinion and preference) data supports the conclu-sion of no recall bias (data reliability). If the null hy-pothesis is rejected, further investigation is requiredto identify pair-wise comparisons that give rise to thedifference. We focus on tests of descriptive statisticsthat are of interest for developing wildlife policy. Al-though each variable is considered individually in theanalyses, the collective statistical findings for all thevariables within each data grouping are considered

before final conclusions are reached regarding ran-dom assignment, recall bias, and datareliability.Data

Surveys were sent to all 900 resident moosehunters who participated in the 1989 hunt. The sur-vey requested socioeconomic characteristics of re-spondents and their characteristics as users ofwildlife, including respondents' sex, age, education,type of city of residence, and income. Education datawere grouped into 4 categories: less than highschool, high school diploma to some college educa-tion, completion of an associate's degree or bache-lor's degree, and completion of an advanced degree.Origin of residence included 4 categories of humanpopulation: ruralareas, small towns (50,000).Hunter characteristics included whether individualshunted moose with a subpermittee and whether thesubpermittee was a Maine resident (a subpermitteeaccompanies the holder of a moose hunting permitand carries a weapon, but only 1 moose can betaken). Other hunter characteristics includedwhether respondents hunt species other than moose,fish inland waters, trap furbearers, or observewildlife.Hunters were asked to report the number and sexof moose seen and the number of black bear (Ursusamericanus) and white-tailed deer (Odocoileus vir-ginianus) seen during the hunt. Hunters were alsoasked to indicate if they chose not to shoot a mooseduring the hunt and to record, by sex, the number ofmoose they chose not to shoot. The Maine Depart-ment of Inland Fisheries and Wildlife uses bullsseen/100 cows seen, calves seen/100 cows seen, andmoose seen/hour hunting to monitor trends in themoose herd. Because of overlaps in habitat of mooseand white-tailed deer in Maine, data are also main-tained on deer seen/hour hunted. If the componentvariables of these ratios do not exhibit recall bias, wecan conclude that the ratios are not subject to recallbias.Each hunter was assigned to a specific zone inwhich they could hunt, and all hunters who killed amoose were required to report to an official checkstation for their moose to be registered. Biologicaldata on sex, age, and physical measurements of themoose were collected at the stations. We asked re-spondents to report the zone in which they huntedand to report the sex and age if they killed a moose.We compared these survey responses with the checkstation datato validate the survey.We collected factual data to document moose


5/10

Recall bias * Mazurkiewicz et al. 143hunting effort as days and hours spent hunting.Hunters were asked to report travel time from theirhome to the zone in which they hunted, number oftrips they made during the hunt, moose hunting ex-penditures, and percent of these expenditures theymade in Maine.

Opinion data included hunters' rating the qualityof their moose hunts on an integer scale rangingfrompoor (1) to excellent (6) and crowding by othermoose hunters on an integer scale ranging from notcrowded (1) to crowded (9). We asked for prefer-ence data regardingthe timing of the moose hunt intwo ways. Huntersfirst were asked their timing pref-erences with options of late September, mid-Octo-ber, or early December. Then respondents were in-formed about the effects of this timing on the qualityof the hunt and were asked to answer the questionagain with response options of late September, mid-October, and no preference (Boyle et al. 1993).TestsWe used chi-square statistics to test the equiva-lency of sample proportions across treatments for bi-naryvariables and variables with multiple categories.Analysisof variance was used to test the equivalencyof sample means across treatments for continuousvariables (e.g., hunter age). When the null hypothe-sis was rejected for any variable, a Duncan multiplerange test was applied to determine which treat-ments differed. A nonparametric Kruskal-Wallisestwas used for continuous variables that had a spike ofobservations at zero (e.g., moose sightings). All testswere conducted with a = 0.10.

ResultsMore than 90% of deliverable surveys were com-pleted and returned, with 97% responding fromGroup Al, 82% from Group A2, 92%from Group B1,

and 93% rom Group C2. The lower response rate forGroup A2 was expected because some respondentssimply reject the second survey or do not take thetime to complete it. In the data analyses we only in-cluded Group A responses for individuals who com-pleted the test and retest.Respondent characteristicsSurveyrespondents were primarilymale (82%),av-eraged 40 years old, had a high school degree (74%)resided in a ruralarea or a small town, and had an av-erage household income of $29,794 (Table 1). Nearlyall respondents (97%) hunted moose with a subper-mittee; 96%of subpermittees were Maine residents.The majorityof moose hunters (72%)hunted in Maine

every year. (Only resident moose hunters were in-cluded in these analyses). A majority hunted gameother than moose, fished open water, ice fished, andobserved wildlife. Only 6% of moose hunters saidthey trapped in Maine.No differences were found among the groups forany of the socioeconomic characteristics; trappingwas the only hunter characteristic that differedamong groups. Because few moose hunters trappedin Maine, this difference may have been a statisticalanomaly. We concluded that hunters were assignedrandomly to experimental groups and respondentcharacteristics did not change between the first andsecond applications of the survey instrument. Thisevidence suggested that the datawere suitable for in-vestigation of recall bias and data reliability.Hunt attributesHunters observed an average of 1.3 calves, 2.5cows, 2.0 bulls, and 1.2 unclassified moose whilehunting in their assigned zone (Table 2). The null hy-pothesis of no difference among groups was rejectedonly for unclassified moose. The pattern of re-sponses across groups did not indicate recall bias.Nearly half of the moose hunters chose not toshoot a calf or a cow, while about 25%chose not toshoot a bull (Table 2). The null hypothesis of no dif-ference in the sample proportions was not rejectedfor any of these variables. Nearly all hunters (93%)killed a moose. Of moose taken, hunters reported86% were adult and 66% were bulls. Check stationdata recorded 92% of resident hunters killing amoose, 89%getting an adult, and 61%getting a bull.The success rate was within the 90%confidence in-terval for survey data (93% CI = 1.4), but the per-cent of hunters getting an adult or a bull were out-side the 90% confidence interval for survey data(86% CI = 2.0; 66% CI = 2.7). The differences, how-ever, were not large. No statistical differences wereidentified for these data across the experimentalgroups.Hunters observed an average of 3.4 deer and 0.2bears while moose hunting (Table 2). The null hy-pothesis of no difference in average observationswas not rejected for deer, but was rejected for bear.Differences existed between treatment C2 and theother 3 treatments, indicating possible over-report-ing recall error. This finding could indicate recallbias that does not occur in the test-retest group re-spondents who may have anchored on their initialresponses when responding to the replication of thesurvey.Recall of the zone hunted closely followed (X2 =13.61, 15 df, P = 0.56) the allocation of hunters to


6/10

144 WildlifeSocietyBulletin1996, 24(1):140-148Table 1. Socioeconomic and personal characteristicsof resident moose hunterssurveyed for recall bias and data reliability in Mainefollowing the 1989 moose hunt.

Variables A1 A2 B1 C2 All groups Test statisticsSocioeconomic characteristics:Sex (%)Male

nAge (years)xSE

nEducation(%)No high school degreeHigh school degreeCollege degreeAdvanced degreenResidence (%)

RuralSmall townSmall cityLargecitynIncome ($)

xSEnHunter characteristics:Hunter moose with subpermittee (%)n

Subpermittee is Maine resident (%)nHunt in Maine every year (%)nHunt other than moose (%)nOpen water fish (%)nIce fish (%)nTrap(%)nObserve wildlife (%)

n

85113381.511325511751133642220103

27,7421,502103

991149596751159211575115641151011560115

83113381.51132156176113284722

310330,2661,956103

961149596731159411578115591151211559115

81 81 82 2 = 1.26547 136 909 P = 0.74400.75472660104547354118

650530,094868505

965479650772551895527855261552555257552

411.41362661114136353923

312529,8801,544125

971339612774137921378313765137413760137

400.590926581339093442204836

29,7946498369790896

82672918909197991962919691958919

F = 0.79P = 0.50

X2 = 26.7P= 0.18

X2 = 18.4P = 0.24

F = 0.48P 0.70X2 = 3.46P= 0.32X = 0.95P = 0.81X2 = 5.66P = 0.93X2 = 3.38P = 0.34X2= 2.92P= 0.402 = 1.10P= 0.78X2 =11.59P = 0.01X2 = 0.64P = 0.89

zones. Except for the southeast zone, numbers fellwithin 90%confidence intervalsfor hunters' recall ofthe zones in which they hunted.Hunters traveled an average of 3.3 hours to theirassigned hunting zones, took an average of 1.8trips from their homes, and averaged 2.4 days(15.7 hours) hunting (Table 3). No differenceswere observed for these variables. Moose huntersspent an average of $423 for their hunts, with 97%being spent in Maine (Table 3). The null hypothe-sis of no differences in the proportions of expen-ditures made in Maine was rejected. Differencesoccurred between Groups Al and A2 and between

Groups Al and C2. This pattern of differences didnot indicate a recall bias (i.e., the null hypothesisof no difference was not rejected for Groups B1and C2).Opinion and preferenceThe majority of respondents rated their hunts asexcellent and did not feel crowded by other moosehunters (Table 4). The null hypothesis of no differ-ence was not rejected for rating of the hunt, but wasrejected for crowding. Reported crowding washigher for Group A than Groups B1 and C2, but thispattern did not indicate unreliable data because


7/10

Recall bias * Mazurkiewicz et al. 145Table2. Hunter electivity,harvest, nd observationsfbig gamewhilemoose hunting s reported yafter the 1989 moose season in Maine. residentmoosehuntersurveyed

Variables Al A2 B1 C2 All groups Test statisticsMooseobserved:Calves

SEnCowsxSEnBullsxSEnUnidentifiedxSEnHunters hat chose not to shoot:Calves %)nCows(%)nBulls %)nKilleda moose (%)

11Adult %)nBull(%)17Otherbig gameobserved:DeerSEnBearSEn

Groups Al and A2, and Groups B1 and C2, did notdiffer.The majority of moose hunters preferred mid-Oc-tober for the moose hunt, regardless of whether theywere simply asked for their preference without infor-mation or were told how timing influences the qual-ity of a hunt (Table 4). These sample proportions dif-fered among groups for the question without infor-mation and did not differ for the question withinformation. Given that the proportion of hunters fa-

voring mid-October exceeds 70% for all groups forthe question without information, the differenceidentified is of no practical importance for settinghunting policy.

DiscussionThe null hypothesis of no difference amonggroups was not rejected for 12 out of the 13 vari-ables in the hunter characteristics data. This weightof evidence led us to conclude that respondentswere assigned randomlyto experimental groups andthat the data satisfactorily investigate recall bias andreliability.The null hypothesis of no difference was not re-

jected for 3 of 4 variables measuring hunter obser-vations of moose and for the 3 hunter selectivityvariables. All 4 harvest variables and 5 of 6 huntereffort variables did not differ among groups. The

1.30.21092.40.31091.90.21111.50.4109

471134411226113921138410364104

3.60.81110.20.06113

1.40.21092.50.31091.90.21111.80.4109

371133611228113921139110365104

3.40.61110.20.05113

1.30.15482.50.25512.10.15521.00.1545

445444254522544935488550767510

3.30.25530.10.02551

1.40.11352.50.21351.90.11351.20.2132

471354113424136961368712963130

3.70.41340.30.08135

1.30.19012.50.19042.00.19091.20.1895

449074190624907939108684266848

3.40.29090.20.02912

X2 = 2.15P= 0.54

X2 = 3.72P= 0.29

X2= 1.98P= 0.58

X2 = 8.22P = 0.04

X2= 3.21P= 0.36X= 1.86P= 0.60X = 2.26P= 0.52X= 1.98P= 0.58X2= 3.69P= 0.30X = 0.77P= 0.86

F = 0.27P= 0.84

X2 = 8.26P - 0.04--


8/10

146 Wildlife Society Bulletin 1996, 24(1):140-148Table 3. Traveltime, effort and hunting expenditures reportedby resident moose hunterssurveyed after the 1989 Maine moose season.

Variables A1 A2 B1 C2 Allgroups TeststatisticsTravel time to zone (hours)x 3.1 3.0 3.4 3.3 3.3 F= 1.69SE 0.2 0.2 0.2 0.2 0.1 P= 0.17n 112 112 545 134 903Number tripsx 1.8 1.9 1.7 2.0 1.8 F= 1.18SE 0.1 0.1 0.1 0.3 0.1 P= 0.32n 107 107 534 133 881Days huntedx 2.4 2.5 2.4 2.3 2.4 F = 0.36SE 0.1 0.1 0.1 0.1 0.05 P= 0.78n 113 113 548 135 909Hours huntedx 17.1 17.3 15.8 14.0 15.7 F= 1.35SE 1.5 1.5 0.6 1.1 0.5 P= 0.26n 112 112 546 132 902Total expenditures ($)x 378 410 435 425 423 F = 0.85SE 30 39 15 28 12 P = 0.47n 110 110 548 136 904Expendituresin Maine (%of Total)x 95 99 98 97 97 F= 2.10SE 1.8 0.9 0.5 1.1 0.4 P= 0.10n 106 106 539 132 883iii~~~~.iiiii iiii

preponderance of evidence supports the conclu-sion that recall bias was not present. Any time alarge number of statistical tests are conducted,there is a probability that a small number of hy-potheses will be rejected purely by chance. Moreimportantly, only 1 of the rejections resulted in a

Table 4. Opinion about the hunt and preference for hunt timing reported by resident moose hunters surveyed after the 1989 Mainemoose hunt.Variables A1 A2 B1 C2 Allgroups Teststatistics

Ratingof hunta5x 4.6 4.6 4.7 4.8 4.7 2= 19.52SE 0.1 0.1 0.1 0.1 0.1 P 0.19n 114 114 549 135 912Crowdingb

x 2.8 2.8 2.4 2.4 2.5 x2 = 36.96SE 0.2 0.2 0.1 0.1 0.1 P = 0.04n 113 113 553 137 916Timing of hunt without information (%)cLateSeptember 23 25 16 25 19 x2 =10.84Mid-October 73 70 79 73 76 P = 0.09EarlyDecember 4 4 5 2 4n 114 114 549 134 911Timing of hunt with information(%)cLateSeptember 20 26 16 21 19 2 = 8.04Mid-October 74 68 80 73 76 P= 0.23No Preference 6 6 4 6 5n 113 113 548 136 910

a Rated on integer scale from poor = 1 to excellent = 6.b Rated on integer scale of not crowded = 1 to crowded = 9.cWithout information representsresponses from hunters who were not informed about implications of season timing to quality tohunt. With informationwas response after briefing on implications.


9/10

Recall bias * Mazurkiewicz et al. 147pattern of pair-wise comparisons that might sup-port recall bias. The bear observation data indi-cated possible recall decay, suggesting that thisproblem might occur for low incidence events onhunting trips that are incidental to the primary ac-tivity. Other differences appeared to be statisticalanomalies.These tests for recall bias implicitly assume,based on the results of the Westat, Inc. (1989)study, that shorter periods of recall reduce recallbias. The Westat study simply compared samplestatistics and did not have actual observations ofhunter and angler activities to confirm this suppo-sition. In our study, comparisons with unpublishedfield data maintained by the Maine Department ofInland Fisheries and Wildlife provided evidencethat hunter recall for a 4-month period, althoughnot perfect, approximated actual behavior in thefield.For opinion-preference data, the differenceamong groups for crowding did not indicate unreli-able databecause average crowding was not differentfor Groups Al and A2 and for Groups Bi and C2. Itsimply appeared that respondents in GroupA experi-enced higher crowding than did hunters in GroupsBI and C2. Ourfindings indicate that reliable hunteropinion and preference datacan be collected that areuseful for developing wildlife management policiesand regulations.

ImplicationsThe message from our research is clear: 4-monthrecall for a unique 1-week hunt does not result in re-call bias. In contrast, Westat, Inc. (1989) found evi-dence of recall bias in a general survey of huntingrequiring annual recall. These studies begin to es-tablish conditions regarding recall period and sur-

vey applications where recall bias is or is not a con-cern. Other researchers should lengthen the recallperiod beyond 4 months and investigate applica-tions to a common activity with frequent participa-tion over an extended time (e.g., deer hunting) toidentify the conditions where recall bias begins tooccur.Our study was robust in use of control groupsthat allowed for identification of significant effectsthat may simply be statistical anomalies. The largenumber of variables provided extensive evidenceto consider in judging real differences among studygroups. Further investigations should include con-

trol groups and investigate all data collected by thesurvey instrument. For any individual surveywhere recall bias is of concern, it is possible tostratify the sample as we have done to test for recallbias. This possibility also holds for concerns re-garding the reliability of opinion and preferencedata. The next step in investigating reliability is toconsider opinion and preference data where the is-sue of concern is more complex and less familiartosurvey respondents (e.g., habitat management is-sues), and the period between the test and retest isextended.

Acknowledgments. The co-authors submitted themanuscript and addressed the reviewers' commentsin the memory of the lead author, StuartMazurkie-wicz, who died in an automobile accident. This re-search was financed by the Maine AgriculturalandForest Experiment Station and the Maine Depart-ment of Inland Fisheries and Wildlife. This publica-tion is MaineAgriculturaland Forest Experiment Sta-tion Publication No. 1892.

Literature citedALLEN,M.J., ANDW. M. YEN. 1979. Introduction to measurement

theory. Brooks Cole Publishing,Monterey,Calif. 310pp.BOYLE, K. J., R. L. DRESSLER,A. G. CLARK, AND M. F. TEISL. 1993.

Moose hunter preferences and setting season timings. Wildl.Soc. Bull. 21:498-504.CHU, A., D. EISENHOWER,M. HAY, D. MORGANSTEIN,J. NETER, AND J.WAKSBERG.1992. Measuringthe recall error in self-reportedfishing and hunting activities. J. Offic. Stat. 5:13-39.DILLMAN,. 1978. Mailand telephone surveys: the total designmethod. John Wiley and Sons, New York,N.Y. 325pp.GEMS,B., D. GOSH,ANDR. HITLIN. 1982. A recall experiment: im-pact of time on recall of recreation fishing trips. Am. Stat.As-soc. 1982 Proc. on survey res. methods:372-375.KEALY,M. J., M. MONTGOMERY,NDJ. F. DORIDIO. 1988. Reliabil-ity and predictive validity of contingent valuation: Does thenature of the good matter? J. Envir. Econ. Manag. 19:244-263.LOOMIS,. B. 1989. Test-retest reliabilityof the contingent valua-tion method: a comparison of general population and visitorresponses. Am.J. Agric. Econ. 71:76-84.U.S. FISHANDWILDLIFEERVICENDU.S. BUREAUFTHECENSUS.1988.1985 national survey of fishing, hunting, and wildlife-associ-ated recreation. U.S. Gov. Printing Off., Washington D.C.137pp.U.S. FISH AND WILDLIFESERVICEAND U.S. BUREAU OF THECENSUS. 1993.1991 national survey of fishing, hunting, and wildlife-associ-ated recreation. U.S. Gov. Printing Off., Washington D.C.124pp.Westat, Inc. 1989. Investigation of possible recall/reference pe-riod bias in nationalsurveys of fishing, hunting, and wildlife-as-

sociated recreation. Rep. to the U.S. Dep. Inter., U.S. FishWildl. Serv., Off. Fed. Aid. 154pp.


10/10

148 Wildlife Society Bulletin 1996, 24(1):140-148StuartMazurkiewicz (photo) received his B.S. from the Universityof New Hampshire. He was a graduate student in the Departmentof Resource Economics and Policy at the University of Maine atthe time this research was conducted. Stuart died in an automo-bile accident priorto submittingthis paper to the Wildlife SocietyBulletin. We at the Bulletinextend our sympathy to Stuart's fam-ily, colleagues and friends and want to express our gratitude forthe opportunity to publish this interesting and informative re-search.

Associate Editor: Peyton

recall bias and reliability of survey data-moose hunting

Documents