Journal of Experimental Psychology:Learning, Memory, and Cognition2001, Vol. 27, No. 6, 1487-1490

Copyright 2001 by the American Psychological Association, Inc.0278-7393/O1/S5.00 DOI: 10.1037*0278-7393.27.6.1487

Are You Sure You Forgot? Feeling of Knowing in Directed Forgetting

Ali I. Tekcan and Melis AkturkBogazici University

One significant issue in metamemory is how variables increasing memorability affect metamemory.Previous research has produced inconsistent results. The effect of directed forgetting on the magnitudeand accuracy of feeling-of-knowing (FOK) judgments was investigated. Participants were presented withword pairs, some to be remembered and some to be forgotten, and then were asked to recall all targetwords regardless of initial instructions. For unrecalled items, they were asked to give FOK judgmentsabout performance in a future memory task: a cued stem-completion task (Experiment 1) or a recognitiontest (Experiment 2). This encoding manipulation increased both the memory performance and themagnitude of FOK judgments. However, no such effect on the accuracy of FOK judgments was observed.

The main purpose of this study was to address the effects ofencoding conditions on feeling-of-knowing (FOK) judgments. Afew studies have investigated this issue with episodic memorytasks, such as paired-associate learning (e.g., Lupker, Harbluk, &Patrick, 1991; Nelson, Leonesio, Shimamura, Landwehr, & Na-rens, 1982; Schacter, 1983). The main question in these studies hasbeen how the strength of encoding relates to metamemory.

In studies with paired-associated learning, it has been consis-tently shown that the degree or the depth of initial encoding leadsto an increase in object-level memory performance, that is,in recall and recognition (e.g., Carroll & Nelson, 1993; Lupkeret al., 1991; Nelson et al., 1982; Schacter, 1983; Schwartz &Metcalfe, 1992). However, the effects of such manipulations onmetamemory measures have been less clear. In studies on FOKjudgments, there are two metamemory measures of interest: themagnitude of FOK judgments regarding performance in a futurecriterion task that is generally less challenging than free recall andthe accuracy of these judgments in predicting the actual perfor-mance in that task. Most studies have consistently demonstrated aneffect on the magnitude but not the accuracy of these judgments.Using a levels-of-processing manipulation, Lupker et al. (1991)found that deeper processing led to better performance in the initialcued-recall task and in the criterion task (cued stem-completiontask). Intensity and accuracy of FOK judgments were also higherfor items that received deeper processing. In another experiment,Lupker et al. (1991) obtained the same pattern of results withstudy-time manipulation.

Similarly, Nelson et al. (1982) and Carroll and Nelson (1993)showed that overlearning influenced measures of memory andmetamemory in the same manner. For instance, Nelson et al.(1982) found that intensity as well as accuracy of FOK judgmentswas higher for items that had been overlearned 1 week before.However, results from some studies (Schacter, 1983; Schacter and

Ali 1. Tekcan and Melis Akturk, Department of Psychology, BogaziciUniversity, Istanbul, Turkey.

We thank Berivan Ece for her help in collection and analysis of data.Correspondence concerning this article should be addressed to Ali t.

Tekcan, Department of Psychology, Bogazici University, Bebek-80815,Istanbul, Turkey. Electronic mail may be sent to tekcanal@boun.edu.tr.

Worling, 1985) contradicted these findings. In both of these stud-ies, increased study time for cue-target pairs (1.5 vs. 5 s) led to anincrease in the frequency of positive FOK judgments but had nosuch effect on accuracy.

Thus, there is abundant evidence that encoding manipulationshave similar effects on explicit object-level measures of memoryand FOK intensity for better-studied materials (for an exception,see Schwartz & Metcalfe, 1992). However, results regarding theeffects of encoding manipulations on FOK accuracy areinconsistent.

The purpose of this study was to provide further data on theeffects of encoding conditions on the magnitude and accuracy ofFOK judgments by use of another well-established paradigm, thedirected-forgetting procedure. Lupker et al. (1991) suggested thatone reason why Schacter (1983) failed to find an effect of in-creased study time on FOK accuracy may be that such a differencemay emerge only if the encoding manipulations lead to a sizabledifference in initial task performance. Moreover, as Carroll andNelson (1993) demonstrated, effects of overlearning become ap-parent when the manipulation is done within subjects. Thus, wereasoned that directed forgetting was an appropriate manipulationfor addressing the effect of initial learning, especially with regardto changes in the magnitude and accuracy of FOK judgments.

In a typical directed-forgetting study with the item method,participants are presented with a list of words, some of which theyare instructed to remember and some of which they are instructedto forget. Then, when they are instructed to remember both theto-be-remembered (TBR) and the to-be-forgotten (TBF) items,recall level for TBR items is higher (Basden, Basden, & Gargano,1993; Bjork and Woodward, 1973; MacLeod, 1989; MacLeod,1999; Mungan & Peynircioglu, 1999). There is agreement thatwhen the item method is used, the TBR-TBF difference stemsfrom selective rehearsal of the TBR items (Basden et al., 1993;Johnson, 1994; MacLeod, 1999).

Experiment 1

In Experiment 1, we explored how the magnitude and theaccuracy of FOK judgments may change as a function of adirected-forgetting procedure, a manipulation that has been shownto consistently increase the memorability of items.

1487

1488 TEKCAN AND AKTORK

Method

Participants. A total of 80 Bogazici University students participated inthe experiment for extra credit in their introductory psychology course.Data from 12 participants were eliminated because the gamma correlation,used to gauge FOK accuracy, could not be calculated for these data. Thedata reported here are based on 68 participants (43 women and 25 men).The mean age of the participants was 19.9 years (SD = 1.49).

Design and procedure. The independent variable (instructions to for-get or remember) was manipulated as a within-subject variable. Partici-pants were tested individually. The instructions followed those of MacLeod(1989). Each participant was presented with 60 word pairs followed byinstructions to either remember or forget a specific word pair. Participantswere told that the letter R indicated that they would have to remember thatword and the letter F indicated that they could forget that word because itwould not be on the test. They were given a practice session with a wordlist consisting of six pairs.

In the experiment, participants were shown each word pair for 1 s andthen received instructions (either to remember or forget) for 3 s. In the testphase, participants were asked to remember the target word for each cueword regardless of the initial instructions. For each target word they failedto remember, they were asked to make a FOK judgment by thinking aboutthe following question: "Would you be able to find the correct word if thefirst two letters of the target word were provided?" They were asked toindicate their FOK judgment on a scale from 1 to 6, where 1 indicatedcertainly no and 6 indicated certainly yes. After participants completed thecued-recall task and FOK judgments, they were given the cued stem-completion task, in which all the cue words as well as the first two lettersof each target word were given. Participants were asked to remember asmany of the target words as possible. Participants worked at their own paceduring the initial cued-recall and cued stem-completion tasks.

Materials. A total of 120 words were selected from a word-frequencydatabase in Turkish, constructed by basically following die KuJera-Francis(1967) methodology (Goz, in press). All were high-frequency nouns (100or more per million) with a length of five or six letters. Sixty word pairswere formed by randomly matching the words. Each word pair was typedin 18-point uppercase letters on 7- by 12-cm index cards. Separate cardswere prepared for the instructions (Remember or Forget), and Turkishequivalents of the letters FFFFF and RRRRR were typed on cards.

Two more sets of cards were prepared, one for the cued-recall test andone for the cued stem-completion test. Each card for the cued-recall testhad the cue word followed by a dash. The cards for the stem-completiontask were the same except that after the dash, the first two letters of thetarget word were also provided. The numbers of letters of the second wordwhich were missing were shown with lines. The orders of the word pairswere different in the three sets.

The study list consisted of 60 word pairs. The first and the last four ofthe pairs served as buffers. Instructions to remember and forget wererandomly assigned to the remaining 52 word pairs, leading to 26 TBRand 26 TBF word pairs. The instruction status of the words was counter-balanced so that each word pair was TBR for half of the participants andTBF for the remaining half. Buffer word pairs were always followed byinstructions to remember.

Results

Two experimenters carried out data collection. Since the pre-sentation during the study was done manually, we checkedwhether there were any experimenter effects. There was no dif-ference between the data collected by these two experimenters onany of the memory or metamemory measures.

The means and standard errors of the means for different mea-sures are given in Table 1.

Table 1Memory and Feeling-of-Knowing (FOK) Measures forTo-Be-Remembered (TBR) and To-Be-Forgotten (TBF)Words in Experiments 1 and 2

Parameter

RecallMedian FOKOverall stem completionConditional stem completionGamma

RecallMedian FOKOverall recognitionConditional recognitionGamma

M

Experiment

.382.93

.51

.25

.21

Experiment

.323.30

.77

.67

.19

Type

TBR

SEM

1

.030.18

.02

.02

.07

2

.020.19

.02

.02

.07

of words

M

.102.09

.22

.17

.21

.072.33

.59

.57

.23

TBF

SEM

.010.14

.02

.01

.06

.020.16

.02

.02

.05

Note. Overall stem completion and overall recognition refer to perfor-mance in the criterion test for all items, whereas conditional stem comple-tion and conditional recognition refer to performance for nonrecalled itemsin the initial recall test. Recall, stem completion, and recognition aremeasured in proportions. FOK is measured on a 6-point scale. Gammacorrelation can take on values between - 1 and 1.

Recall. A clear directed-forgetting effect was observed onrecall performance; participants remembered more TBR wordsthan TBF words, ?(67) = 13.30, p < .0001.

FOK ratings. Median FOK ratings for each participant werecalculated for the TBR and TBF items separately. Overall, partic-ipants gave higher FOK ratings for TBR words than for TBFwords, t(61) = 6.28, p < .0001. It is also worth noting that medianFOK ratings for both types of items were moderate (2.09 and 2.93on a 6-point scale).

Cued stem-completion test. Participants were provided witheach cue word as well as the first two letters of the correspondingtarget word. The effect of encoding instructions on the criterionperformance was analyzed in two ways. First, we looked at overallstem-completion performance regardless of whether the wordswere recalled or not in the initial cued-recall test. There was a cleareffect of directed forgetting; a higher proportion of TBR wordsthan of TBF words was completed, ;(67) = 13.13, p < .001,reflecting a difference closely resembling that in the initial recalltest. Second, when we looked at stem-completion performanceconditional upon nonrecall in the initial task, a higher proportion ofTBR words than of TBF words again was completed successfully,f(67) = 4.36, p < .001.

FOK accuracy. We used Goodman and Kruskal's gamma asthe measure of FOK accuracy (Nelson, 1984; see also Nelson,1996, and Wright, 1996, in response to Schraw, 1995). The gammavalues for TBR and TBF words were virtually identical,f(67) = 0.01, p > .10. In addition, although both of the gammacorrelations were above chance in predicting criterion perfor-mance, 1(67) = 2.95 for TBF words and 2.81 for TBR words, p <.01 for both, the gamma values were quite low for both types of

FEELING OF KNOWING IN DIRECTED FORGETTING 1489

items, as is generally the case in paired-associate learning (e.g.,Lupker et al., 1991; Schwartz & Metcalfe, 1992).

Experiment 2

The lack of a difference in FOK accuracy between TBR andTBF words in Experiment 1 might have been partly attributable tothe nature of the criterion test used. To address this possibility, wereplicated Experiment 1 by using a recognition test as the criteriontask.

Method

Participants. A total of 73 Bogazici University students participated inthe experiment for extra credit in their introductory psychology course.Data from 7 participants were eliminated because the gamma correlationcould not be calculated for these data. The mean age of the remaining 66participants (47 women and 19 men) was 19.58 years (SD = 1.30).

Materials, procedure, and design. The 60 word pairs used in Experi-ment 1 were used in this experiment as well. The only change was that thecriterion task in this experiment was a forced-choice recognition test withfive alternatives comprising the correct target, two semantically relatedlures, and two phonologically related lures.

Results

The means and standard errors of the means for different mea-sures are given in Table 1.

Recall. There was a reliable directed-forgetting effect; partic-ipants recalled a higher proportion of TBR words than of TBFwords, f(65) = 10.89, p < .001.

FOK ratings. Participants gave higher FOK ratings for unre-called TBR words than for unrecalled TBF words. This differencewas significant, f(65) = 4.91, p < .001.

Recognition test. Overall, more TBR words than TBF wordswere recognized correctly, f(65) = 10.08, p < .001. When thesame analysis was carried out conditionalized upon nonrecall inthe initial recall test, a higher proportion of TBR words than ofTBF words again was recognized, t{65) = 4.50, p < .001.

FOK accuracy. There was no difference between the gammacorrelations for TBR words and TBF words, f (65) = 0.43, p > . 10.Although gamma values for both types of words were again low,they were above chance in predicting recognition performance,K65) = 4.36 for TBR items and 2.66 for TBF items, p < .01 forboth.

General Discussion

We showed that directed-forgetting manipulation influencedthe intensity but not the accuracy of FOK judgments. Thefinding that an initial encoding manipulation affected the mag-nitude of FOK is consistent with previous literature (e.g., Lup-ker et al., 1991; Nelson et al., 1982) as well as with the twoframeworks attempting to account for how FOK judgments aremade. The accessibility account (Koriat, 1993), which suggeststhat FOK judgments are based on the amount of partial infor-mation and speed with which that information comes to mind,would predict higher FOK judgments for TBR items than forTBF items because more partial information would becomeavailable for better-learned information. The cue-familiarityaccount (e.g., Reder & Ritter, 1992; Schwartz & Metcalfe,

1992), which suggests that FOK judgments are based on howfamiliar the cue is, would predict the same pattern because theTBR cue words would be more familiar than the TBF cue wordsas a result of the encoding manipulation.

The main factor for differences in FOK accuracy as a functionof encoding manipulations in some previous studies was that theaccuracy was virtually nil for less-well-encoded items and abovechance for better-encoded items (Carroll and Nelson, 1993; Lupkeret al., 1991; Nelson et al., 1982). Our results, along with those ofSchacter (1983) and Schacter and Worling (1985), do not supportthis pattern. One potential reason may be that encoding manipu-lations need to have a large effect on initial memory performancefor FOK accuracy to be affected (Koriat, 1993; Lupker et al.,1991). Although this reason is a potential factor, our findingsappear to eliminate this explanation because the directed-forgetting instructions had a strong effect on initial cued-recall aswell as criterion performance tests. It is obviously still possiblethat the TBR-TBF difference in our study did not exceed someunidentified threshold. However, the amount of difference in var-ious memory measures in this study seems similar to those ob-tained in studies that reported differences in FOK accuracy (e.g.,Lupker et al., 1991).

References

Basden, B. H., Basden, D. R., & Gargano, G. J. (1993). Directed forgettingin implicit and explicit memory tests: A comparison of methods. Journalof Experimental Psychology: Learning, Memory, and Cognition, 19,603-616.

Bjork, R. A., & Woodward, A. E., Jr. (1973). Directed forgetting ofindividual words in free recall. Journal of Experimental Psychology, 99,22-27.

Carroll, M., & Nelson, T. O. (1993). Effect of overleaming on the feelingof knowing is more detectable in within-subject than in between-subjectdesigns. American Journal of Psychology, 105, 227-235.

GOz, 1. (in press). Yazih Turkge'nin kelime sikhgi sozliigu [Word frequencydictionary of written Turkish]. Ankara, Turkey: Turk Dil Kurumu.

Johnson, H. M. (1994). Processes of successful intentional forgetting.Psychological Bulletin, 116, 274-292.

Koriat, A. (1993). How do we know that we know? The accessibility modelof the feeling of knowing. Psychological Review, 100, 609—639.

Ku£era, H., & Francis, W. N. (1967). Computational analysis of present-day American English. Providence, RI: Brown University Press.

Lupker, S. J., Harbluk, J. L., & Patrick, A. S. (1991). Memory for thingsforgotten. Journal of Experimental Psychology: Learning, Memory, andCognition, 17, 897-907.

MacLeod, C. M. (1989). Directed forgetting affects both direct and indirecttests of memory. Journal of Experimental Psychology: Learning, Mem-ory, and Cognition, 15, 13-21.

MacLeod, C. M. (1999). The item and list methods of directed forgetting:Test differences and the role of demand characteristics. PsychonomicBulletin and Review, 6, 123-129.

Mungan, E., & Peynircioglu, Z. F. (1999). Unutmaya yonlendirmeninbellek iizerinde niteliksel etkileri [Qualitative effects of directed forget-ting on memory]. Turk Psikoloji Dergisi, 14, 1-12.

Nelson, T. O. (1984). A comparison of current measures of the accuracy offeeling-of-knowing predictions. Psychological Bulletin, 95, 109—133.

Nelson, T. O. (1996). Gamma is a measure of the accuracy of predictingperformance on one item relative to another item, not of the absoluteperformance on an individual item: Comment on Schraw (1995). Ap-plied Cognitive Psychology, 10, 257-260.

Nelson, T. O., Leonesio, R. J., Shimamura, A. P., Landwehr, R. F., &

1490 TEKCAN AND AKTURK

Narens, L. (1982). Overlearning and the feeling of knowing. Journal ofExperimental Psychology: Learning, Memory, and Cognition, 8, 279-288.

Reder, L. M., & Ritter, F. E. (1992). What determines initial feeling ofknowing? Familiarity with question terms, not with the answer. Journalof Experimental Psychology: Learning, Memory, and Cognition, 18,435-451.

Schacter, D. L. (1983). Feeling of knowing in episodic memory. Journal ofExperimental Psychology: Learning, Memory, and Cognition, 9, 39-54.

Schacter, D. L., & Worling, J. R. (1985). Attribute information and thefeeling-of-knowing. Canadian Journal of Psychology, 39, 467—475.

Schraw, G. (1995). Measures of feeling-of-knowing accuracy: A new lookat an old problem. Applied Cognitive Psychology, 9, 321-332.

Schwartz, B. L., & Metcalfe, J. (1992). Cue familiarity but not targetretrievability enhances feeling-of-knowing judgments. Journal of Exper-imental Psychology: Learning, Memory, and Cognition, 18, 1074—1083.

Wright, D. B. (1996). Measuring feeling of knowing: Comment on Schraw(1995). Applied Cognitive Psychology, 10, 261-268.

Received February 23, 2001Revision received April 6, 2001

Accepted May 21, 2001 •