Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=ncen20

Download by: [University of Patras], [Dr Lambros Messinis] Date: 24 November 2015, At: 01:47

Journal of Clinical and Experimental Neuropsychology

ISSN: 1380-3395 (Print) 1744-411X (Online) Journal homepage: http://www.tandfonline.com/loi/ncen20

Age and education adjusted normative data anddiscriminative validity for Rey’s Auditory VerbalLearning Test in the elderly Greek population

Lambros Messinis, Grigorios Nasios, Antonios Mougias, Antonis Politis,Petros Zampakis, Eirini Tsiamaki, Sonia Malefaki, Phillipos Gourzis &Panagiotis Papathanasopoulos

To cite this article: Lambros Messinis, Grigorios Nasios, Antonios Mougias, AntonisPolitis, Petros Zampakis, Eirini Tsiamaki, Sonia Malefaki, Phillipos Gourzis & PanagiotisPapathanasopoulos (2015): Age and education adjusted normative data and discriminativevalidity for Rey’s Auditory Verbal Learning Test in the elderly Greek population, Journal ofClinical and Experimental Neuropsychology, DOI: 10.1080/13803395.2015.1085496

To link to this article: http://dx.doi.org/10.1080/13803395.2015.1085496

Published online: 20 Nov 2015.

Submit your article to this journal

View related articles

View Crossmark data

Age and education adjusted normative data and discriminative validityfor Rey’s Auditory Verbal Learning Test in the elderly Greek populationLambros Messinisa, Grigorios Nasiosb, Antonios Mougiasc, Antonis Politisd, Petros Zampakise,Eirini Tsiamakia, Sonia Malefakif, Phillipos Gourzisg and Panagiotis Papathanasopoulosa

aNeuropsychology Section, Department of Neurology, University of Patras Medical School, Patras, Greece; bHigher EducationalInstitute of Epirus, Ioannina, Department of Speech and Language Therapy, Ioannina, Greece; cAlzheimer Center of the GreekPsychogeriatric Association “Nestor,” Athens, Greece; dDepartment of Psychiatry, Eginition Hospial, School of Medicine, Universityof Athens, Athens, Greece; eDepartment of Radiology, University of Patras Medical School, Patras, Greece; fDepartment ofMechanical Engineering & Aeronautics, University of Patras (statistics), Patras, Greece; gDepartment of Psychiatry, University ofPatras Medical School, Patras, Greece

ABSTRACTRey’s Auditory Verbal Learning Test (RAVLT) is a widely used neuropsychological test toassess episodic memory. In the present study we sought to establish normative anddiscriminative validity data for the RAVLT in the elderly population using previouslyadapted learning lists for the Greek adult population. We administered the test to 258cognitively healthy elderly participants, aged 60–89 years, and two patient groups (192with amnestic mild cognitive impairment, aMCI, and 65 with Alzheimer’s disease, AD).From the statistical analyses, we found that age and education contributed significantlyto most trials of the RAVLT, whereas the influence of gender was not significant.Younger elderly participants with higher education outperformed the older elderlywith lower education levels. Moreover, both clinical groups performed significantlyworse on most RAVLT trials and composite measures than matched cognitively healthycontrols. Furthermore, the AD group performed more poorly than the aMCI group onmost RAVLT variables. Receiver operating characteristic (ROC) analysis was used toexamine the utility of the RAVLT trials to discriminate cognitively healthy controlsfrom aMCI and AD patients. Area under the curve (AUC), an index of effect size, showedthat most of the RAVLT measures (individual and composite) included in this studyadequately differentiated between the performance of healthy elders and aMCI/ADpatients. We also provide cutoff scores in discriminating cognitively healthy controlsfrom aMCI and AD patients, based on the sensitivity and specificity of the prescribedscores. Moreover, we present age- and education-specific normative data for individualand composite scores for the Greek adapted RAVLT in elderly subjects aged between 60and 89 years for use in clinical and research settings.

ARTICLE HISTORYReceived 6 March 2015Accepted 17 August2015Revised 28 July 2015

KEYWORDSRey’s auditory verballearning test; greek; episodicmemory; normative dataelderly; composite measures;discriminant validity; mildcognitive impairment;alzheimer’s disease.

Neuropsychological tests are essential for a reliableand valid assessment of cognitive functioning andhave been widely recognized for this purpose (Lezak,Howieson, Bigler, & Tranel, 2012). However, diagnos-tic decisions and rehabilitation planning can only bemeaningful if culture-, language-, and demographic-specific normative data are available.

The lack of normative data for commonly usedneuropsychological tests in Greece has led neurop-sychologists working in Greece to either developculture- and language-specific new tests (Folia &Kosmidis, 2003; Kosmidis, Vlahou, Panagiotaki, &

Kiosseoglou, 2004; Vlahou et al., 2013) or collectnormative data for commonly used neuropsycho-logical tests developed in other countries (Aretouli& Kosmidis, 2006; Argirokastritou, Samanda, &Messinis, 2005; Economou, 2003; Giannakou &Kosmidis, 2006; Konstantinopoulou et al., 2011;Messinis, Lada, et al., 2006; Messinis, Tsakona, &Papathanasopoulos, 2006; Papathanasiou,Messinis, Georgiou, & Papathanasopoulos, 2014;Vlahou & Kosmidis, 2002; Zalonis et al., 2009).

In an effort to contribute to the development ofappropriate culture-, language-, and demographic-

CONTACT Lambros Messinis, lmessinis@upatras.gr Neuropsychology Section, Department of Neurology, University of Patras MedicalSchool, Rio, 26504 Patras, Greece.

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY, 2015http://dx.doi.org/10.1080/13803395.2015.1085496

© 2015 Taylor & Francis

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

specific neuropsychological test norms for theGreek population, in 2007 we presented normativedata for the adult population of a commonly usedneuropsychological measure of verbal learning andepisodic memory, the Rey Auditory VerbalLearning Test (RAVLT; Messinis, Tsakona,Malefaki, & Papathanasopoulos, 2007; Rey, 1964).The RAVLT has been translated and normed inseveral languages and cultures (e.g., FerreiraCorreia & Campagna Osorio, 2014; Lannoo &Vingerhoets, 1997; Lee, 2003; Van der Elst, VanBoxtel, Van Breukelen, & Jolles, 2005) with normsshowing significant variance regarding their rela-tionship with demographic variables.

Difficulty with memory and learning is one ofthe most frequent complaints of individuals withneuropsychological disorders and may contributesignificantly to disability in activities of daily livingand functional outcome (Lezak et al., 2012; Malec& Thompson, 1994; Schoenberg et al., 2006).Memory complaints in outpatient settings alsoappear to be the most frequent reason for neurop-sychological referral (Lezak et al., 2012). Moreover,many common neurological and psychiatric disor-ders produce deficits in memory processes(Papathanasiou et al., 2014; Schoenberg et al.,2006; Vlahou et al., 2013).

Decline in episodic memory constitutes one ofthe core neuropsychological symptoms, particu-larly in elderly individuals at risk of developingAlzheimer’s disease or during the preclinical stagesof dementia known as mild cognitive impairment(MCI; Lezak et al., 2012; Speer et al., 2014). Due toa significant increase in elderly patients’ subjectivememory complaints (Zygouris & Tsolaki, 2015)and reports (Mitchell & Shiri-Feshki, 2009;Summers & Saunders, 2012) that patients withMCI have elevated rates of conversion to demen-tia, the objective and accurate assessment of episo-dic memory in this population is crucial fordiagnostic purposes.

The RAVLT (Rey, 1964; Messinis et al., 2007;Schmidt, 1996) is a multitrial verbal learningtest that is preferred to other tests under con-ditions of limited assessment time and in casesin which clinicians are solely interested in list-learning abilities and wish to dissociate theseabilities from conceptual organization abilitiessuch as those tapped by the California VerbalLearning Test (Lezak et al., 2012). The testaffords the ability of a person to encode, con-solidate, store, and retrieve verbal information

and is usually applied using a five-trial presen-tation of a 15-noun word list (List A; Trials 1,2, 3, 4, 5;presentation rate of one word persecond), a single presentation of a 15-nounword interference list (List B; Trial 6), twopostinterference recall trials (one immediate,Trial 7, and one delayed, Trial 8, D , rangingfrom 20 to 45 minutes, most commonly around30 minutes), and a recognition trial (Trial 9, R)of 50 words containing the target words of ListsA and B and 20 distractor words phonetically orsemantically similar to those in Lists A and B(Lezak et al., 2012; Schmidt, 1996).

Although several cognitive processes can beextracted by analyzing performance of the pre-viously mentioned individual trials, it has beensuggested that computing composite scores,which aggregate several trials, may provide apurer index of a specific cognitive process (Vakil,Greenstein, & Blachstein, 2010). Furthermore,composite scores may better reflect theoreticalindices of memory than raw scores (Lezak et al.,2012). These scores are more informative than thescores for the single learning trials, as each scorealone does not reflect learning (Lezak et al., 2012).More specifically, learning ability may be betterreflected by two such composite measures/scoresgenerated from the RAVLT. The first is Trial 5minus Trial 1, which would reflect the learningrate. Performance on this RAVLT composite mea-sure was found to be deficient in mild cognitiveimpairment (MCI) and Alzheimer’s disease (AD)patients (Nordlund et al., 2007). The second is thetotal score of all five learning trials, which reflectstotal acquisition/learning. Normative data areavailable for this measure from various studiescovering a broad age range for both genders(Geffen, Moar, O’Hanlon, Clark, & Geffen, 1990;Schmidt, 1996; Vakil et al., 2010).

Another factor that may influence RAVLT per-formance is proactive and retroactive interference.Proactive interference takes place when previouslylearned material contributes negatively to acquisi-tion or recall of new information. Trial B alonecannot reflect interference without being com-pared to Trial 1, which serves as a baseline.Older age increases susceptibility to proactiveinterference (Vakil et al., 2010). Retroactive inter-ference occurs when subsequently presentedmaterial negatively influences the recall of pre-viously learned material (Lezak et al., 2012;Vakil et al., 2010).

2 L. MESSINIS ET AL.

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

The long-term retention and forgetting rate mayalso be assessed by the RAVLT by testing the recallof List A (Trial 8) after a 20–45-minute delayperiod. However, Vakil and Blachstein (1997) sug-gest that viewing Trial 8 (D) as reflecting delayedrecall or retention ability is insufficient if it is notcompared to Trial 5, which serves as the baselinefor the number of words learned. Thus, a moreaccurate measure of delay recall is Trial 5 minusTrial 8. Patients with MCI and Alzheimer’s diseasewho perform low on Trial 8 have significant defi-cits on long-term retention (Gunther, Holtkampa,Jolles, Herpetz-Dahlamanna, & Konrad, 2004).Similarly, for the recognition measure (Trial 9;retrieval efficiency), without a comparison withthe preceding delayed recall trial (Trial 8), inter-pretation of the score on Trial 9 is problematic.Thus, it is recommended that the number of thewords in Trial 8 is subtracted from that in Trial 9in order to reflect retrieval efficiency (Vakil et al.,2010). Difficulties in retrieval are common inpatients diagnosed with MCI (Broder, Herwig,Teipel, & Fast, 2008) and Alzheimer’s disease(Gainotti, Marra, & Villa, 2001). EarlyAlzheimer’s disease patients recall few words onTrial 1 and reach to about 6 words by Trial 5. Theyalso have particular difficulty recalling words aftera delay with distraction and recognize about twomore words than they recall, with many intrusionerrors (Lezak et al., 2012).

The RAVLT has shown adequate utility in dis-criminating cognitively healthy elderly frompatients with MCI and preclinical AD (Estevez-Gonzalez, Kulisevsky, Boltes, Otermin, & Garcia-Sanchez, 2003; Siraly et al., 2015). Moreover, false-alarm responses in the RAVLT recognition trials,associated with more widespread temporal brainarea function, may be a helpful marker for earlydiagnosis of MCI (Zeidman et al., 2008). TheRAVLT has also shown adequate discriminatoryability when administered repeatedly in differen-tiating subjects with MCI from those who wereinitially diagnosed with MCI but subsequentlyreverted to normal (Antuono et al., 2007).

A large body of research shows that demo-graphic variables, most notably age and years offormal education, and less consistently intelligencelevel and gender, influence performance on theRAVLT. The influence of age on RAVLT perfor-mance has been reported by numerous studies andhas gained wide acceptance as the most influentialdemographic variable on RAVLT performance

(Crossen & Wiens, 1994; Ferreira Correia &Campagna Osorio, 2014; Speer et al., 2014;Geffen et al., 1990; Messinis et al., 2007;Uchiyama et al., 1995; Van der Elst et al., 2005).

In the adult population, two segments can bedistinguished: the younger adult age groups (20 to59) years and the older adult age groups (60 to 90)years. For the younger adult segment, RAVLTperformance for younger subjects is not as differ-entially sensitive as performance obtained by theolder age groups, possibly suggesting different sto-rage capacities and strategy utilization in the twoadult segments (Lezak et al., 2012; Strauss,Sherman, & Spreen, 2006). Utilizing a Hebrewversion of the AVLT, Vakil and Blachstein (1997)noted modest changes in AVLT performance inparticipants below the age of 60 compared toincreasingly reduced recall in participants over 60years old. Healthy elderly subjects, in comparisonto younger ones, also tend to show greater forget-ting rates of words at the end of the list duringdelayed recall, suggesting that older subjects relymore on short–lived memory processes—that is,immediate recall of the last words on the list—than do younger subjects (Lezak et al., 2012).

The influence of gender, years of formal educa-tion, and intellectual level appears less consistentacross various studies (Lezak et al., 2012; Schmidt,1996; Strauss et al., 2006). More specifically, Geffenet al. (1990) reported better performance inwomen adult participants than in men in the agerange of 16–86 years. Lannoo and Vingerhoets(1997) noted that women, young adults, and per-sons with higher educational levels outperformedmen, older adults, and individuals with fewer yearsof formal education. Generally, when studiesreport differences in performance due to influ-ences of gender, women tend to outperform menon the recall trials, but not on the recognition trials(Geffen et al., 1990; Lannoo & Vingerhoets, 1997;Miatton, Wolters, Lannoo, & Vingerhoets, 2004;Speer et al., 2014; Strauss et al., 2006). Other stu-dies, however (Harris, Ivnik, & Smith, 2002; Vander Elst et al., 2005), reported better performanceof women on the recognition trials in addition torecall trials. The recognition trials measure howmuch was learned, regardless of the efficiency ofspontaneous retrieval (Strauss et al., 2006). Therecognition process allows for the distinctionbetween problems with registration and storagefrom those of inefficient recall (Powell & Cripe,1991). Normally, if the patient’s problem is simply

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY 3

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

difficulty in retaining new information, then therecognition score will be little better than the delayrecall trial score (Strauss et al., 2006). There are,however, reports in the literature of nonsignificantcontributions of gender to RAVLT performance(Ferreira Correia & Campagna Osorio, 2014;Forrester & Geffen, 1991; Mitrushina, Boone,Razani, & D’Elia, 2005; Savage & Gouvier, 1992).

Regarding the specific contribution of intelli-gence levels to RAVLT performance, the trend isthat recall is better in persons with higher intelli-gence levels (Strauss et al., 2006; Vakil &Blachstein, 1997). Steinberg, Bieliauskas, Smith,Ivnik, and Malec (2005) reported the influence ofintellectual functioning on RAVLT performanceand presented age and intelligence level adjustednormative data for older adults (Mayo’s OlderAmericans Normative Studies, MOANS).Steinberg et al. (2005) further reported strongercorrelations between RAVLT trials and full-scaleIQ (FSIQ) at moderate levels of intelligence.

The literature regarding the specific influence offormal education on RAVLT scores has been con-tradictive. There are studies that have reportedbetter performance in persons with higher educa-tional levels (Lannoo & Vingerhoets, 1997;Miatton et al., 2004; Van der Elst et al., 2005),while others have not reported a significant con-tribution of education to RAVLT performance(Mitrushina et al., 2005; Mitrushina, Satz,Chervinsky, & D’Elia, 1991; Wiens, McMinn, &Crossen, 1988). In addition, certain authors notedthat education does not account for RAVLT per-formance beyond that which is accounted for byintelligence level (Strauss et al., 2006).

As stated previously, in 2007, normative data(Messinis et al., 2007) were published for theRAVLT using newly adapted learning lists for theGreek adult population. One of the main limita-tions in that normative study was the lack of stra-tified subgroups for elderly participants over theage of 60. Elderly participants normally show amore distinguishable pattern of performancedecline with advancing age, and it would havebeen preferable to have included narrower group-ings. Another limitation was the unavailability ofdata for persons aged 80 and over.

In order to overcome these limitations, weassessed a relatively large group of healthy elderlyadults and provide normative data based on demo-graphic characteristics specific to the Greek cultureand language for the RAVLT in this older

population. We also examined the test’s validityin differentiating the performance of healthyelderly persons from that of patients with episodicmemory deficits, by assessing large samples ofpatients diagnosed with amnestic mild cognitiveimpairment (aMCI) and Alzheimer’s disease(AD) and comparing their performance to anage-, gender-, and education-matched cognitivehealthy control group. Moreover, we explored thediagnostic utility of the test through receiveroperator characteristic (ROC) analyses in patientswith aMCI and AD and provide cutoff scores,stratified by age, in discriminating cognitivelyhealthy controls from aMCI and AD patients,based on the sensitivity and specificity of the pre-scribed scores.

Method

Participants

Two hundred and fifty eight (258) cognitivelyhealthy elderly native Greek speakers (161 femalesor 62.4%) recruited primarily from southwesternGreece and Athens, took part in the present studyvoluntarily and after providing written consent fortheir participation. Healthy participants were con-tacted/approached and if interested were invited totake part in the study by the staff (neurologists,psychiatrists, psychologists, and neuropsycholo-gists) of the outpatient Neuropsychology Unit,Department of Neurology, University of PatrasMedical School and the Alzheimer Center of theGreek Psychogeriatric Association “Nestor,” inAthens, Greece. These participants wereapproached mainly through senior citizen centersand our outpatient clinics, as stated above.

Healthy participants were aged between 60 and89 years (age: M = 74.21 years, SD = 7.71; level ofeducation: M = 9.85 years, SD = 4.59). Exclusioncriteria for the healthy participants were a historyof psychiatric, neurological, or cardiovascular dis-orders or of substance abuse or dependence(including alcohol and benzodiazepine abuse),any other medical condition (including hearingimpairment) that might affect neuropsychologicalperformance, and non-native speakers of theGreek language. We further excluded from thestudy elderly adults who on initial testing obtainedscores of less than 27 on the Greek validated ver-sion of the Mini-Mental State Examination(Fountoulakis, Tsolaki, Chantzi, & Kazis, 2000), a

4 L. MESSINIS ET AL.

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

brief screening measure for global cognitive defi-cits. Elderly adults who met a current Diagnosticand Statistical Manual of Mental Disorders, FifthEdition (DSM–5; American PsychiatricAssociation, 2013), diagnosis of late life depression(LLD) were also excluded.

We also examined a group of 192 patientsrecruited at either of the previously mentionedclinical settings that were diagnosed with mildcognitive impairment (MCI) amnestic subtype(aMCI). For the diagnosis of aMCI, we used mod-ified criteria proposed by Petersen et al. (1999)including: (a) subjective memory complaint bythe patient or his/her caregiver; (b) normal generalcognitive function as defined by a score of ≥24 onthe Greek version of the MMSE (Fountoulakiset al., 2000); (c) normal activities of daily living(ADLs) judged both clinically and on an ADLscale; (d) Clinical Dementia Rating (CDR) ratingof 0.5; (e) objective memory decline below the 16thpercentile on neuropsychological testing; (f)absence of major psychopathology; and (g) absenceof significant white matter ischemia on magneticresonance imaging (MRI). We excluded partici-pants from this group who met a current DSM–5(American Psychiatric Association, 2013) diagnosisof major neurocognitive disorder (MND), had adependence on any drug or alcohol, or sufferedfrom any other medical condition that might affectneuropsychological performance, and non-nativespeakers of the Greek language. aMCI patientshad a mean age of 75.17 years (SD = 6.05) and amean level of education of 10.14 years (SD = 4.38).

We further examined a third group of 65 patientsrecruited at either of the previously mentioned clin-ical settings that were diagnosed with Alzheimer’sdisease (AD). These patients were diagnosedaccording to the National Institute of Neurologicaland Communicative Disorders and Stroke and theAlzheimer’s Disease and Related DisordersAssociation (NINCDS–ADRDA) guidelines forprobable AD (McKhann et al., 1984). These guide-lines have established neuropsychological criteria,with the cutoff score for impairment as being the2nd percentile or lower in three cognitive domains,including orientation, memory, attention, language,perceptual skills, praxis, reasoning, and functionalstatus. In addition, for this group an abnormalMini-Mental State Examination (MMSE) score of≤23 was required and a CDR rating of at least 1. Wefurther included participants in this group who meta current DSM–5 (American Psychiatric

Association, 2013) diagnosis of major neurocogni-tive disorder (MND) due to Alzheimer’s disease.AD patients had a mean age of 75.55 years (SD =6.38) and a mean level of education of 10.60 years(SD = 4.24).

Procedure

Healthy participants were tested individually bypsychologists at either of the previously men-tioned clinical settings. These participants wereinitially screened through a standardized inter-view at the beginning of the testing session bythe project staff clinical neuropsychologist andphysician (neurologist or psychiatrist), in orderto exclude those with health problems or otherexclusion criteria as described above. They werealso administered the Greek validated MMSE(Fountoulakis et al., 2000) and the Greek vali-dated version of the Geriatric Depression Scale(Fountoulakis et al., 1999). Psychologists takingpart in the testing of participants had been inten-sively trained in the administration procedures ofvarious neuropsychological measures, includingthe RAVLT, by doctoral-level clinical neuropsy-chologists. Participants were then assessed usingGreek adapted lists of the RAVLT (Messinis et al.,2007). These adapted lists were based partly onthe original English lists presented in Schmidt(1996, p. 71; List A, learning list, and List B,interference list) and Lezak, Howieson, andLoring (2004, p. 423). The original Englishwords presented in the above lists were initiallytranslated without change to the Greek language.From these two original English word lists weretained 10 of the original words from List Aand seven of the original words from List B(interference trial) as presented in Schmidt(1996, p. 71). The remaining words used for thedevelopment of the Greek lists were new wordsadapted for the Greek language. The word itemson these newly formed Greek lists were evaluatedfor consistency on the following dimensions: Allwords were two- or three-syllable concrete nouns;there were no obvious semantic or phonetic asso-ciations or similarities between words in the samelist; all were common words that are normallyacquired in Greek-speaking persons with rela-tively low levels of education; and all words hadfrequent occurrence in the Greek language. Theprobability of the occurrence of the word incommon usage in the Greek language was

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY 5

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

ascertained using the Institute for Language andSpeech Processing Greek Corpus (Hatzigeorgiuet al., 2000). Using the above criteria, formequivalence was established between the two newlists. These two new lists yielded comparablemean scores for the various trials of the RAVLT,as there were no significant differences betweenthe two lists on the matching consistency dimen-sions, as described previously (Messinis, Tsakona,et al., 2006). The original Lists A and B (Lezaket al., 2004, p. 423; Schmidt, 1996, p. 71) and thetwo new Greek lists (A1 and B1) in the order thatthe words were read to participants are presentedin Table 1. The administration procedure usedwas the one originally followed by Geffen et al.(1990), as referred to in Schmidt (1996) andLezak et al. (2004). We must note, however, thatafter participants completed Trial B (interferencetrial), we administered other nonverbal neuropsy-chological tests (Symbol Digit Modalities Test;Benton Visual Retention Test; Color Trails TestParts 1 and 2) for approximately 25 minutes inorder to avoid interference with previouslylearned verbal material (Argirokastritou et al.,2005; Messinis, Lyros, Georgiou, &Papathanasopoulos, 2009, Messinis, Malegiannaki,Christodoulou, & Papathanasopoulos, 2011). Afterthis 25-min delay period, we asked participants torecall the words learned in trials 1–5 (delay recalltrial) and then administered the recognition trialasking the patients to identify the target words ofList A.

Patients diagnosed with aMCI or AD were alsotested individually by psychologists at either of thepreviously mentioned clinical settings. They hadbeen previously diagnosed by a multidisciplinaryconsensus group composed of neurologists, psy-chiatrists, neuropsychologists, and speech patholo-gists utilizing the criteria mentioned above. Theywere initially administered the Greek validatedMMSE (Fountoulakis et al., 2000) and the Greekvalidated version of the Geriatric Depression Scale(Fountoulakis et al., 1999). They were than admi-nistered a brief neuropsychological battery com-prising the RAVLT (Messinis et al., 2007), tests ofexecutive functioning and psychomotor speed,including the color trails test (only Part 1;Messinis et al., 2011), the Symbol DigitModalities Test (Argirokastritou et al., 2005), atest of visuospatial ability, the Clock DrawingTest (Bozikas, Giazkoulidou, Hatzigeorgiadou,Karavatos, & Kosmidis, 2008), and a test assessingnonverbal memory and visuoconstruction, theBenton Visual Retention Test (Messinis et al.,2009). These two patient groups underwentRAVLT testing by the same administration proce-dure as that described previously for the healthyelderly group.

Statistical analyses

We initially examined our data visually to deter-mine whether the distributions met normalityrequirements. All data points that were consideredoutliers or extreme outliers were excluded from theanalyses. Then, we calculated the main descriptivestatistics for all studied variables and tested groupsseparately. The normality assumption of our datawas also tested using the Kolmogorov–Smirnovtest for normality. Generally, the normalityhypothesis was rejected in almost all cases so thenonparametric Kruskal–Wallis test was used toexamine whether the samples of the tested groupsoriginate from the same distribution (healthy par-ticipants, aMCI patients, and AD patients). Posthoc pairwise comparisons between the testedgroups were also provided. Furthermore, in orderto examine the potential contribution of the demo-graphic variables age, gender, and years of formaleducation on the performance of the differenttrials of the RAVLT in the full sample of healthyelderly adults, nonparametric Spearman partialcorrelation coefficients were computed, and theirstatistical significance was also tested. Finally, ROC

Table 1. The original RAVLT (Lists A and B) and greek lists(A1 and B1).Original list Greek lists

List Aa List BaList A1b,c

listList A1word

List B1b,c

listList B1word

Drum Desk Day Mera Desk ThranioCurtain Ranger Curtain Kourtina Number ArithmosBell Bird Train Treno Bird PouliCoffee Shoe Coffee Kafes Shoe PapoutsiSchool Stove School Skolio Child PediParent Mountain Parent Yoneas Mountain VounoMoon Glasses Hand Heri Water NeroGarden Towel Garden Kipos Book VivlioHat Cloud Hat Kapelo Cloud SinefoFarmer Boat Farmer Agrotis Boat VarkaNose Lamb Nose Miti Bridge YefiraTurkey Gun Street Dromos Woman YinekaColor Pencil Color Hroma Pencil MoliviHouse Church House Spiti Button KoubiRiver Fish Door Porta Body Soma

aOriginal list (Lezak et al., 2004; Rey, 1964; Schmidt, 1996).bGreek lists (Messinis et al., 2007).cOrder in which the Greek list was read to subjects and at a pre-sentation rate of one word per second.

6 L. MESSINIS ET AL.

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

curve analyses were utilized in order to calculatesensitivity and specificity on RAVLT variables foreach of the two patient groups in order to evaluatewhether these variables differentiate cognitivelyhealthy elderly from these two pathological condi-tions. In all the statistical hypotheses tested thelevel of statistical significance was set at α = .05.Statistical analyses were conducted using the sta-tistical package SPSS 22.0 for Windows (SPSS,Chicago, IL) and also the statistical programminglanguages R Version 3.1.

Results

Initially, in order to examine thoroughly the shapeof the studied variables, we created boxplots for allthe variables for each patient group. As noted inFigure 1, there is no common pattern in the shapesof the distributions of the studied variables. In mostof the cases, there is a discrepancy from normality,and also the distributions are skewed either posi-tively or negatively. However, no floor or ceilingeffect is present, with the exception of the RAVLTdelay recall trial in the group with AD who show afloor effect. Moreover, the nonparametric Spearmancorrelation coefficients of RAVLT trials (variables)showed most items to be significantly correlatedwith each other (Table 2).

Effect of demographic variables on RAVLTtest performance

In order to examine the potential contribution ofdemographic variables age, gender, and years offormal education on the performance of the differ-ent trials of the RAVLT in the full sample of healthyelderly adults, nonparametric Spearman partial cor-relation coefficients were computed, and their sta-tistical significance was tested (see Table 3). In

general, age and education significantly influencedthe performance on almost all RAVLT variablesexamined. The older elderly with a lower educationlevel performed worse than the younger elderly withhigher education levels. Moreover, gender did notsignificantly influence performance on most of theRAVLT variables, with the exception of the delayrecall and retention trials.

Age and education stratified normative data

Given the significant influence of education andage on performance to most of the RAVLT trialsrevealed by the analyses, and in order to generatenormative data for the Greek elderly populationover the age of 60, we grouped our sample intothree age groups—60–69, 70–79, and 80–89 years—and also stratified our sample based on the levelof education, so as to reflect actual school require-ments in Greece (compulsory education is nineyears): 1–9 years and above 9 years. In Table 4we present normative data for the Greek healthy

Table 2. Correlations among RAVLT variables in healthy elderly.Variable RAVLT 1 RAVLT 5 RAVLT B RAVLT D RAVLT R Total learning Learning rate Retention Retrieval

RAVLT 1 — .571* .547* .512* .418* .782* –.040 .081 .015RAVLT 5 — .416* .714* .522* .882* .775* .194* –.042RAVLT B — .323* .335* .522* .093 .218* .010RAVLT D — .607* .774* .519* –.426* –.346*RAVLT R — .604* .343* –.105 .460*Total learning — .499* .092 –.036Learning Rate — .168* –.090Retention — .428*Retrieval —

Note. RAVLT = Rey Auditory Verbal Learning Test; RAVLT 1 RAVLT Trial 1; RAVLT 5 = RAVLT Trial 5; RAVLT B = RAVLT Trial B (interference trial);RAVLT D = RAVLT Trial D (delay recall trial); RAVLT R = RAVLT Trial R (recognition trial); total learning = Σ Trials 1–5; learning rate = Trial 5 –Trial 1; retention = Trial 5 – Trial D; retrieval efficiency = Trial R – Trial D.

*p < .01.

Table 3. Contributions of education, age, and gender toRAVLT individual trial and composite scores.Variable Age Education Gender

RAVLT 1 –.3993* .2447* .0431RAVLT 5 –.5068* .2733* .0092RAVLT B –.4961* .3615* –.1317RAVLT 7 –.2581* .3349* .1791RAVLT 8 Rec –.2434* .3541* .1971*Learning rate –.2394* .1414* .0636Retention –.2485 –.0920 –.2215*Retrieval –.0210 .0599 .0059Total learning –.4681* .3378* .0620

Note. RAVLT = Rey Auditory Verbal Learning Test; RAVLT 1≥ RAVLTTrial 1; RAVLT 5 = RAVLT Trial 5; RAVLT B = RAVLT Trial B (inter-ference trial); RAVLT 7 = RAVLT Trial 7 (immediate recall); RAVLT D= RAVLT Trial D (delay recall trial); RAVLT R = RAVLT Trial R(recognition trial); total learning = Σ Trials 1 –5; learning rate =Trial 5 – Trial 1; retention = Trial 5 – Trial D; retrieval efficiency =Trial R – Trial D.

*p < .05.

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY 7

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

elderly population stratified by age and educationlevel.

Group differences in RAVLT performance

In order to determine the validity of the RAVLT indiscriminating the performance of the patientgroup from that of healthy elderly participants,we conducted comparisons on the RAVLT out-come parameters between the three groups(aMCI, AD patients, and cognitive healthy partici-pants). The three groups were matched on genderratio, χ2(2) = 5.653, p = .059, level of education, F(2, 503) = 1.024, p = .360, and age, F(2, 503) =1.615, p = .200. Utilizing the Kruskal–Wallis non-parametric test, we noted a significant main groupeffect on Trial 1, χ2(2) = 37.445, p < .001, trial 5, χ2

(2) = 67.465, p < .001, trial B, χ2(2) = 25.707, p <.001, trial D, χ2(2) = 133.529, p < .001, and trial R,χ2(2) = 58.148, p < .001. Moreover, we noted asignificant main group effect for the compositestotal learning (Σ Trials 1–5), χ2(2) = 69.48, p <.001, learning rate (Trial 5 – Trial 1), χ2(2) = 41.3 p< .001, and retention (Trial 5 – Trial D), χ2(2) =48.28, p < .001.

Descriptive statistics for each trial and compo-site of the RAVLT are presented in Table 5, includ-ing pairwise comparisons between aMCI andhealthy elderly, between AD and healthy elderly,and between aMCI and AD patients. These pair-wise comparisons revealed significant differencesbetween aMCI patients and the healthy elderly onmost trials of the RAVLT, with the exception ofTrial B (interference trial) and the compositelearning rate. AD patients differed significantlyfrom the healthy elderly on all the RAVLT trialsexamined. Further, comparison of the two clinicalgroups showed that the performance of ADpatients was significantly different from that ofaMCI patients on all individual trials. However,nonsignificant differences between the two clinicalgroups were noted on the composites retentionand retrieval efficiency.

Clinical utility

We explored the clinical utility of the test througha receiver operator characteristic (ROC) curve inorder to calculate sensitivity and specificity.Specifically, we examined the utility of theRAVLT individual trials and composites to discri-minate cognitively healthy controls from aMCITa

ble4.

Normativedata

forhealthyelderly

stratifiedby

ageandlevelo

feducation.

Age(in

years)

Education(in

years)

NI

VB

DR

TLLR

Ret

Retr

60–69

1–9

234.5(1.3)

9.1(2.2)

3.7(1.5)

6.6(2.1)

11.2

(2.8)

36.2

(7.5)

4.5(2.4)

2.5(1.3)

4.5(2.6)

9+70

5.3(1.7)

10.8

(2.0)

5.2(1.4)

8.1(2.3)

12.2

(2.4)

41.5

(8.3)

5.5(2.0)

2.7(2.0)

4.0(1.8)

70–79

1–9

503.8(1.8)

7.3(2.1)

3.7(1.2)

4.5(2.6)

9.4(3.3)

29.0

(8.2)

3.4(1.9)

2.71.7

4.9(2.6)

9+36

4.7(1.8)

8.3(2.6)

5.0(1.3)

7.4(2.8)

12.0

(2.5)

36.9

(8.9)

3.6(3.2)

1.03.9

5.3(3.0)

80–89

1–9

533.1(1.0)

6.8(1.7)

2.4(1.4)

4.2(2.0)

8.2(1.8)

25.9

(6.73)

3.5(1.3)

2.4(2.0)

4.0(3.1)

9+26

3.3(0.6)

7.0(1.3)

2.9(0.9)

5.1(1.4)

9.1(1.5)

24.7

(5.0)

3.6(1.4)

1.8(1.2)

3.8(1.6)

Note.Means;stand

arddeviations

inparentheses.RA

VLT=ReyAu

ditory

VerbalLearning

Test;I=RA

VLTTrial1;V

=RA

VLTTrial5;B

=RA

VLTTrialB

(interference

trial);D=RA

VLTTrialD

(delay

recalltrial);R=RA

VLT

TrialR

(recog

nitio

ntrial);

TL=totallearning(Σ

Trials1–5);LR=learning

rate

(Trial5

–Trial1

);Ret=retention(Trial5

–TrialD

);Retr=retrievale

fficiency

(TrialR

–TrialD

).

8 L. MESSINIS ET AL.

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

and AD patients. The area under the ROC curve(AUC), an index of effect size and overall diagnos-tic accuracy, showed that in most cases the RAVLTtrials adequately differentiated between the perfor-mance of healthy elders and aMCI/AD patients,although there were some exceptions that are dis-cussed below. We also provide cutoff scores indiscriminating cognitively healthy controls fromaMCI and AD patients, based on the sensitivityand specificity of the prescribed score (seeTable 6).

Discussion

Neuropsychological assessment is essential for thediagnosis of memory and other cognitive deficitsin the elderly and for their effective care and treat-ment. Given the significant increase in the numberof elderly in Greece, which has one of the highestaging rates in Europe (National Statistical Office ofGreece, 2014) the need for standardized neuropsy-chological assessment tools that deliver high-qual-ity information and are specific for culture,language, and demographic variables is urgent.

Despite the widespread use of neuropsychologi-cal measures in clinical and research settings inGreece in the last few years, normative data forcommon neuropsychological assessment measuresespecially for the elderly population remain largelyunavailable. In an effort to contribute towards fill-ing this gap, we generated culture- and language-specific normative data for the Greek elderly popu-lation of a widely used, easily administered measureof verbal learning and episodic memory, theRAVLT (Rey, 1964; Schmidt, 1996), adequatelystratified by demographic variables that contributed

significantly to RAVLT performance on verballearning lists adapted for the Greek adult popula-tion (Messinis et al., 2007). We further provide dataon the test’s clinical utility in discriminating elderlypatients with verbal learning and episodic memorydeficits. In our previous presentation of normativedata for the adult population in Greece (Messiniset al., 2007), we did not adequately stratify ournorms for elderly participants over the age of 60,and data for adults aged 80 and over were notavailable. To our knowledge there have been noother attempts to date to collect normative datafor the RAVLT in the elderly population in Greece.

The RAVLT has gained wide recognition in theaging literature and has been used extensively forthe assessment of multiple memory indices andmemory processes, such as initial recall, consolida-tion, retrieval, recognition, and proactive and ret-roactive interference (Constantinidou et al., 2014;Speer et al., 2014). In general, older adults over theage of 60 perform worse than their younger coun-terparts on the learning trials of the RAVLT(Messinis et al., 2007; Constantinidou et al.,2014). Further, older adults diagnosed with amnes-tic mild cognitive impairment (aMCI), a transi-tional state between healthy aging and dementia,perform in the impaired range on this and othersimilar episodic verbal memory tests(Constantinidou et al., 2014; Estevez-Gonzalezet al., 2003).

The results of the present study revealed thatage accounted for a substantial proportion of thevariance in RAVLT performance favoring youngerelderly participants—that is, a decline in perfor-mance was observed with increasing age in a linearfashion. This finding is consistent with reports in

Table 5. Group comparisons on the RAVLT individual trials and composites.

Variable

Healthy elderly(N = 258)

aMCI patients(N = 192)

AD patients(N = 65)

M SD M SD M SD aMCI/ healthy elderly p AD/ healthy elderly p aMCI/AD p

RAVLT 1 4.24 1.76 3.54 1.53 2.80 1.59 .000 .000 .002RAVLT 5 8.38 2.83 7.39 2.66 5.15 2.31 .000 .000 .000RAVLT B 4.06 2.01 3.74 1.29 2.62 1.92 .171 .000 .000RAVLT D 6.13 3.29 3.82 2.10 1.03 1.07 .000 .000 .000RAVLT R 10.51 3.38 8.70 4.18 6.09 3.30 .000 .000 .000Total learning 33.12 10.15 28.57 9.25 21.07 9.08 .000 .000 .000Learning rate 4.13 2.37 3.81 2.22 2.35 1.76 .401 .000 .000Retention 2.25 2.13 3.58 2.15 4.12 1.89 .000 .000 .236Retrieval 4.43 2.89 4.91 3.08 5.06 3.93 .000 .000 .382

Note. RAVLT = Rey Auditory Verbal Learning Test; aMCI = amnestic mild cognitive impairment patients; AD = Alzheimer’s disease patients; RAVLT1 = RAVLT Trial 1; RAVLT 5 = RAVLT Trial 5; RAVLT B = RAVLT Trial B (interference trial); RAVLT D = RAVLT Trial D (delay recall trial); RAVLT R =RAVLT Trial R (recognition trial); total learning = Σ Trials 1–5; learning rate = Trial 5 – Trial 1; retention = Trial 5 – Trial D; retrieval efficiency =Trial R – Trial D.

Asymptotic significance (two-sided tests); the significance level is .05.

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY 9

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

the literature that age is the most influential demo-graphic characteristic on RAVLT performance(Crossen & Wiens, 1994; Geffen et al., 1990;

Uchiyama et al., 1995; Van der Elst et al., 2005).The finding that age contributes significantly toRAVLT performance is also true for the elderly

Table 6. RAVLT cutoff scores with sensitivity, specificity, PPV, and NPV values stratified by age comparing elderlyparticipants with normal cognition to those with mild cognitive impairment and those with Alzheimer’s disease.Age(years) Variable Group

Cutoff scores (rawscores)

Sensitivity(%)

Specificity(%)

PPV(%)

NPV(%) AUC [95% CI]

60–69 RAVLT 1 NC vs. MCI 4.5 81.6 71.1 47.0 88.7 0.722 [0.622, 0.817]NC vs. AD 3.5 93.3 90.1 40.7 94.9 0.785 [0.643, 0.926]

RAVLT 5 NC vs. MCI 11.5 76.3 38.9 31.2 74.3 0.656 [0.539, 0.774]NC vs. AD 8.5 86.7 84.4 38.9 98.6 0.912 [0.832, 0.992]

RAVLT B NC vs. MCI 5.5 92.1 46.7 38.0 91.7 0.689 [0.592, 0.785]NC vs. AD 5.5 86.9 71.2 20.5 90.2 0.677 [0.533, 0.821]

RAVLT D NC vs. MCI 7.5 78.9 63.3 37.3 78.7 0.676 [0.562, 0.790]NC vs. AD 5.5 100 84.4 51.7 100 0.973 [0.946, 1.000]

RAVLT R NC vs. MCI 11.5 81.6 73.3 44.1 82.6 0.682 [0.573, 0.790]NC vs. AD 11.5 86.7 73.5 23.3 91.9 0.688 [0.576, 0.800]

Total learning NC vs. MCI 40.5 84.2 61.1 37.5 80.4 0.676 [0.566, 0.787]NC vs. AD 36.5 93.3 75.6 31.8 98.4 0.870 [0.756, 0.985]

Learning rate NC vs. MCI 6.5 81.6 45.6 34.8 82.1 0.536 [0.422, 0.649]NC vs. AD 4.5 100 77.8 34.9 100 0.840 [0.765, 0.914]

Retention NC vs. MCI 3.5 78.9 34.4 26.4 62.2 0.466 [0.352, 0.580]NC vs. AD 6.5 80 3.3 11.9 25.0 0.260 [0.124, 0.95]

Retrievalefficiency

NC vs. MCI 4.5 63.2 44.5 32.4 74.1 0.528 [0.411, 0.645]

NC vs. AD 11.5 86.7 3.3 12.7 33.3 0.053 [0.009, 0.097]70–79 RAVLT 1 NC vs. MCI 4.5 78.1 50.6 62.4 58.4 0.601 [0.520, .682]

NC vs. AD 4.5 87.2 60.7 38.0 91.8 0.686 [0.592, 0.780]RAVLT 5 NC vs. MCI 8.5 78.1 43.8 55.3 58.4 0.550 [0.468, 0.632]

NC vs. AD 6.5 83.9 70.8 50.0 92.6 0.795 [0.715, 0.875]RAVLT B NC vs. MCI 4.5 77.1 58.4 60.5 56.0 0.580 [0.498, 0.661]

NC vs. AD 4.5 80.6 57.3 34.7 87.5 0.774 [0.673, 0.874]RAVLT D NC vs. MCI 5.5 85.7 65.2 69.2 76.6 0.707 [0.633, 0.780]

NC vs. AD 3.5 100 80.9 55.4 100 0.934 [0.890, 0.978]RAVLT R NC vs. MCI 10.5 72.4 68.5 64.1 57.1 0.667 [0.591, 0.744]

NC vs. AD 7.5 90.3 83.1 65.1 96.1 0.911 [0.854, 0.967]Total learning NC vs. MCI 35.5 81.9 39.3 57.7 57.8 0.620 [0.541, 0.699]

NC vs. AD 28.5 93.5 79.8 52.7 96.9 0.844 [0.771, 0.918]Learning rate NC vs. MCI 4.5 72.4 44.9 53.6 44.9 0.491 [0.408, 0.574]

NC vs. AD 3.5 77.4 60.7 34.8 86.3 0.702 [0.606, 0.798]Retention NC vs. MCI 5.5 83.8 13.5 51.8 29.2 0.330 [0.254, 0.406]

NC vs. AD 6.5 87.1 7.9 23.9 42.9 0.213 [0.124, 0.303]Retrievalefficiency

NC vs. MCI 6.5 67.6 49.4 56.8 50.7 0.500 [0.418, 0.582]

NC vs. AD 6.5 93.5 49.4 34.9 94.6 0.688 [0.578, 0.798]80–89 RAVLT 1 NC vs. MCI 3.5 61.2 38.0 34.9 54.8 0.506 [0.395, 0.617]

NC vs. AD 2.5 73.7 78.5 45.2 92.5 0.777 [0.628, 0.926]RAVLT 5 NC vs. MCI 7.5 79.6 43.0 37.6 60.5 0.555[0.445, 0.665]

NC vs. AD 5.5 78.9 73.4 41.7 93.5 0.825 [0.683, 0.968]RAVLT B NC vs. MCI 4.5 83.7 8.9 35.0 27.3 0.298 [0.208, 0.388]

NC vs. AD 2.5 78.9 54.4 25.0 89.5 0.620 [0.465, 0.774]RAVLT D NC vs. MCI 4.5 81.6 70.9 59.5 75.8 0.648 [0.548, 0.748]

NC vs. AD 2.5 84.2 77.2 47.1 95.3 0.818 [0.710, 0.926]RAVLT R NC vs. MCI 8.5 65.3 54.5 35.7 58.6 0.502 [0.394, 0.610]

NC vs. AD 10.5 89.5 40.5 23.6 92.3 0.675 [0.531, 0.819]Total learning NC vs. MCI 27.5 77.6 45.6 37.8 60.9 0.528 [0.421, 0.635]

NC vs. AD 20.5 89.5 83.5 40.0 92.1 0.811 [0.678, 0.945]Learning rate NC vs. MCI 4.5 75.5 38.0 39.4 64.7 0.534 [0.427, 0.642]

NC vs. AD 3.5 89.5 51.9 26.6 94.1 0.744 [0.599, 0.889]Retention NC vs. MCI 4.5 73.5 32.9 37.1 58.1 0.326 [0.223, 0.428]

NC vs. AD 4.5 84.2 31.6 20.8 85.7 0.350 [0.219, 0.481]Retrievalefficiency

NC vs. MCI 6.5 73.5 32.9 32.2 47.4 0.408 [0.307, 0.509]

NC vs. AD 8.5 89.5 26.6 20.7 87.5 0.423 [0.262, 0.584]

Note. NC = healthy elderly participants with normal cognition; MCI = amnestic mild cognitive impairment patients; AD = Alzheimer’s diseasepatients;

AUC = area under the curve; CI = confidence interval; PPV = positive predictive value; NPV = negative predictive value;RAVLT = Rey Auditory Verbal Learning Test; RAVLT 1 = RAVLT Trial 1; RAVLT 5 = RAVLT Trial 5; B = RAVLT Trial B (interference trial); RAVLT D =RAVLT Trial D (delay recall trial); RAVLT R = RAVLT Trial R (recognition trial); total learning = Σ Trials 1–5); learning rate = Trial 5 – Trial 1;retention = Trial 5 – Trial D; retrieval efficiency = Trial R – Trial D .

10 L. MESSINIS ET AL.

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

population, with past (Steinberg et al., 2005) andmore recent (Speer et al., 2014) studies providingsuch evidence.

This age-dependent decline in episodic memoryhas been linked to various brain alterations,including decreased information processing speeddue to reductions in brain volume (Raz, 2000).Further, white matter loss, especially in the pre-frontal areas, is associated with deficits in tasksassessing attention, working memory, and execu-tive functions, implying that aging leads to a nat-ural loss of cognitive functions (Speer et al., 2014).

Reports regarding the effects of formal educa-tion on RAVLT performance have been mixed inthe literature. One possible explanation for thisdiscrepancy is that it depends on the amount ofdifference or the actual levels of education beingconsidered. In this study, we found that level ofeducation contributed significantly to most of thetrials and composite scores on the RAVLT.Participants with lower levels of education in theformal Greek educational system (under 9 yearsof education) performed worse than participantswith higher levels of education. Unlike most

studies, as reported by Schmidt (1996), whichhave found relatively weak relationships betweeneducation and RAVLT performance, and suggestthat education is not a substantial demographiccharacteristic contributing to this test, in the pre-sent study we showed that education is a sub-stantial demographic contributor when makingnormative comparisons in older Greek adultsover the age of 60. The present data, as regardsthe influence of education on RAVLT perfor-mance, support our previous study that includedyounger and older Greek adults (Messinis et al.,2007) and numerous other studies that havefound similar results (Constantinidou et al.,2014; Lannoo & Vingerhoets, 1997; Miattonet al., 2004; Van der Elst et al., 2005) and thatincluded older participants up to 86 years old.There are, however, studies that support thenotion of Schmidt (1996) of relatively inconsistentand insignificant contributions of education toRAVLT performance (Mitrushina et al., 2005;Mitrushina et al., 1991; Wiens et al., 1988),including a recent study with older adults (Speeret al., 2014).

Figure 1. Boxplots of the studied variables for the elderly healthy, amnestic mild cognitive impairment patients (MCI), andAlzheimer’s disease (AD) patients. RAVLT = Rey Auditory Verbal Learning Test. To view this figure in color, please visit theonline issue of the Journal.

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY 11

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

In contrast to our previous study (Messiniset al., 2007) that revealed a significant contributionof gender to RAVLT performance in a sample ofyounger and older adults, in this study only therecognition trial and retention composite wasinfluenced by gender. This is in keeping with themajority of the cited literature where the contribu-tion of gender has appeared less consistent(Schmidt, 1996; Steinberg et al., 2005). When aneffect of gender has been noted, the general trendis that women outperform men on the recall trials,but not on the recognition trials (Geffen et al.,1990; Lannoo & Vingerhoets, 1997; Miatton et al.,2004; Strauss et al., 2006), although some studies(Harris et al., 2002; Van der Elst et al., 2005)reported better performance of women partici-pants on the recognition trials in addition to recalltrials.

One possible reason that may explain the lack ofgender dominance favoring females in RAVLTperformance in our elderly sample in the presentstudy compared to our previous study of youngerand older adults (Messinis et al., 2007), whichfound such an effect, is that aging females have areduced neuroprotective influence of female sexhormones (Speer et al., 2014). Moreover, agingmay possibly reduce the known higher verbal abil-ity of women, which has been linked to betterperformance in verbal memory tests (Lezak et al.,2012; Speer et al., 2014).

Regarding the influence of intelligence level toRAVLT performance in the present study, wechose not to examine the contribution of intelli-gence in our elderly Greek sample for two mainreasons: First, when the study was initiated therewere no available standardized tests of intelligencein Greece for adults/elderly. Secondly, if our datawere found to be influenced by intelligence level,and thus stratified by this variable, this wouldrequire that participants tested for verbal learningand memory deficits would have to also completean intelligence test to establish full-scale IQ, beforethe norms could be used adequately. In clinicaloutpatient settings where most of the elderlyGreek with memory problems are assessed, intelli-gence testing, which is highly time consuming,requires specialized training to interpret, and isdifficult to complete, especially in the elderly andthe demented, would not allow for norms stratifiedby intelligence to be used.

As noted previously, a valid assessment of mem-ory function is often essential for diagnosing

memory decline. On clinical grounds, the diagno-sis of preclinical dementia stages relies on thediagnosis of mild cognitive impairment (MCI).The criteria for this diagnosis subsume the pre-sence of subjective memory complaints, especiallythe amnestic MCI subtype, preferably corroboratedby a close informant, and documented by abnor-mal performance on cognitive testing, based ondemographically adjusted normative data(Petersen, Doody, et al., 2001; Summers &Saunders, 2012). In this study, we compared theperformance of elderly healthy elderly frompatients with aMCI and Alzheimer’s dementia.We found that all trials of the test adequatelydifferentiated the performance of aMCI patientsfrom that of the healthy elderly, with the exceptionof the interference trial and the composite measurelearning rate. Our findings regarding the ability ofmost RAVLT trials to differentially diagnose cog-nitively healthy elderly from aMCI patients sup-port previous studies (Estevez-Gonzalez et al.,2003; Zeidman et al., 2008) that found similarresults. However, although the finding regardingthe learning rate may appear unexpected, Broderet al. (2008) note that both the elderly and MCIpatients may show a slower rate of learning andimprovement in tasks of memory and repeatedlearning. In contrast, all trials and composite mea-sures differentiated the performance of ADpatients from that of the healthy elderly. Variousstudies have noted that the RAVLT may help toidentify patients with Alzheimer’s dementia andeven distinguish with a high degree of accuracybetween types of dementia (Estevez-Gonzalezet al., 2003; Ricci, Graef, Blundo, & Miller, 2012;Tierney et al., 1994).

When comparing the two clinical groups, wefound that all individual trials differentiated theperformance of AD patients from that of theMCI group. However, the composite measuresretention and retrieval efficiency were found tobe nonsignificant between the two clinical groups.This finding implies that both clinical groups showrelatively increased rates of forgetting and retrievalefficiency and support the findings of Broder et al.(2008) and Gainotti et al. (2001), who stipulate thatthese deficits are common in both MCI and ADpatients.

Regarding the results of the ROC analyses, wefound that the trials with the highest overall diag-nostic accuracy (AUC), independent of cutoffscores, for all age categories were the delay recall

12 L. MESSINIS ET AL.

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

trial and the total learning composite measure thatdiscriminated healthy elderly from AD patients.Overall, AUCs ranged from poor (i.e., age group60–69 retention and retrieval composites) to veryhigh with excellent discriminatory utility (i.e.,60–69 Trial 5 and Trial D). A large range of sensi-tivities, specificities, positive predictive values(PPVs), and negative predictive values (NPVs)were also recorded, ranging from low to high, forthe various cutoff scores provided. In some casesthe composite scores that were calculated assistedin formulating a more refined differential diagno-sis, although very low AUCs and sensitivities, spe-cificities, NPVs, and PPVs were also noted. In thisrespect, it is suggested that the clinician utilize thecombination of these values in determining theaccuracy of the trial (see Table 6).

An important question arising from our find-ings is what makes performance on the RAVLTsensitive to early decline in aMCI and AD patients.One possible explanation is that medial temporallobe atrophy (MTA; hippocampal–parahippocam-pal formation) seems to be an important anatomi-cal feature of AD and its prodromal stage,amnestic mild cognitive impairment (PeruzzaMarchianni, Figuerdo Balthazar, Cendes, &Pereira Damasceno, 2008). Episodic memory—that is, the ability to acquire explicit information—is affected most in aMCI and AD. In contrast,memory related to events that occurred earlier inlife (past events) remains intact for long periods oftime. This pattern of impairment reflects deficitsmainly in memory processes of encoding and sto-rage, in contrast to retrieval, which remains intactuntil the late stages of AD (Peruzza Marchianniet al., 2008; Ricci et al., 2012). A recent study thatinvestigated the relationship of the functionalmemory processes and medial temporal lobe atro-phy in patients with aMCI and AD (Boon, Melis,Olde Rikkert, & Kessels, 2011) found significantassociations with the encoding and storage pro-cesses, but not with retrieval. This finding impliesthat aMCI and AD patients show a decline inencoding and storage of verbal information thatis related to MTA atrophy. Further, PeruzzaMarchianni et al. (2008) noted significant correla-tions between hippocampal volumes and scores onthe RAVLT, confirming that medial temporal lobestructures are closely related with verbal memoryperformance in normal aging as well as in aMCIand AD patients. It therefore appears that atrophyof the medial temporal lobe regions may predict

the presence of future AD, as hippocampal degen-eration appears to occur before the onset of overtdementia (Estevez-Gonzalez et al., 2003).

In evaluating the generalizability of our find-ings, several potential limitations need to be high-lighted. First, we realize that by not generatingintelligence adjusted normative data we may limitthe interpretive accuracy of the RAVLT in thispopulation. On the other hand, if such intelli-gence adjusted normative data were utilized inGreek settings, the test norms would most prob-ably be ignored in the absence of intelligencetesting availability. Secondly, decreased familiar-ity with neuropsychological assessment proce-dures, which differ from traditional medicalprocedures to which elderly individuals inGreece have become accustomed, may have alsoinfluenced our findings. Examiners were, how-ever, well trained in the administration of theRAVLT and had previous experience with elderlyparticipants. Significant efforts were made inorder to ensure that these older participants cor-rectly understood all administration procedures,therefore minimizing this possible limitation.Another potential limitation concerns the risk ofsampling bias associated with motivation to takepart in this study. It would appear that elderlyGreek individuals willing to participate in thestudy are more motivated and possibly more cur-ious of what a neuropsychological examinationinvolves. Finally, an important limitationimposed upon the present study and other studiesof MCI arises from lack of consensus regardingthe clinical definition of MCI. Although our par-ticipants were selected according to the standarddefinition in the field (Petersen et al., 1999;Petersen, Stevens, et al., 2001), criteria for aMCIrequire mainly memory impairment. However,consistent with the findings in the literature(Constantinidou et al., 2014; Estevez-Gonzalezet al., 2003) and the results obtained by ouraMCI patients on the brief neuropsychologicalbattery (results not reported here), which wasadministered together with the RAVLT, thesepatients are often impaired in other cognitivedomains. A more valid classification of this pre-dementia state may therefore be necessary in thefuture.

Finally, although this study focuses on theunique clinical utility of the RAVLT in diagnosis(e.g., the RAVLT is sensitive to early decline inaMCI and AD patients), there is a large possibility

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY 13

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

that several of the other measures in our neurop-sychological battery would have similar predictivepower (e.g., perhaps RAVLT is related to globaldecline, so many measures would do as well).

Despite these potential limitations to the gener-alizability of the present normative data, the studyprovides a reference point for the neuropsycholo-gical assessment of verbal learning and episodicmemory in elderly Greek adults over the age of60 and extends the available Greek norms toelderly patients 89 years of age. Moreover, thestudy is based on culture- and language-specificlearning lists that prevent Greek clinicians fromhaving to inappropriately rely on English-wordlists and norms for English-speaking or otherpopulations in the elderly.

Even more importantly for clinical practice, thestudy provides evidence that the Greek version ofthe RAVLT adequately distinguishes the perfor-mance of predementia or amnestic MCI patientsfrom that of cognitively healthy elderly, underlin-ing its usefulness in the differential diagnoses ofearly dementia of the Alzheimer’s type. Moreover,the test is able to differentiate among major episo-dic memory decline of AD patients and the moreminor verbal episodic memory impairments ofaMCI patients.

Future research is required in order to collectnormative data for the Greek nondemented oldestold (90+) and in order to establish the diagnosticutility of the Greek learning lists in differentiatingamong Alzheimer’s dementia and other dementiasubtypes for, for example, behavioral variant fron-totemporal dementia.

Disclosure statement

No pontential conflict of intrest was reported by theauthor(s).

References

American Psychiatric Association. (2013). Diagnosticand statistical manual of mental disorders (5th ed.).Arlington, VA: American Psychiatric Publishing.(2013).

Antuono, P., Jones, J., Wu, Z., Xu, G., Franczak, M., &Jiang-Li, S. (2007). Repeated administration ofRAVLT differentiates MCI from MCI that reverts tonormal. Alzheimer’s and Dementia, 3(Suppl.), S138.

Aretouli, E., & Kosmidis, M. H. (2006, July). Culturaldifferences on Wisconsin Card Sorting Test perfor-mance: What do neuropsychological tests really

measure? Paper presented at the 26th InternationalCongress on Applied Psychology, Athens.

Argirokastritou, E., Samanda, T., & Messinis, L. (2005,December). Normative data for the Symbol DigitModalities Test (SDMT) in Greece. Paper presentedat the 10th Pan-Hellenic Congress of Psychology,Ioannina, Greece.

Boon, M. E., Melis, R. J. F., Olde Rikkert, M. G. M., &Kessels, R. P. C. (2011). Atrophy in the medial tem-poral lobe is specifically associated with encoding andstorage of verbal information in MCI and Alzheimerpatients. Journal of Neurological Research. Advanceonline publication. doi:10.4021/jnr18w

Bozikas, V. P., Giazkoulidou, A., Hatzigeorgiadou, M.,Karavatos, A ., & Kosmidis, M. H. (2008). Do age andeducation contribute to performance on the clockdrawing test? Normative data for the Greek popula-tion. Journal of Clinical and ExperimentalNeuropsychology, 30, 199–203.

Broder, A., Herwig, A., Teipel, S., & Fast, K. (2008).Different storage and retrieval deficits in normalaging and mild cognitive impairment: A multinomialmodeling analysis. Psychology and Aging, 23, 353–365.

Constantinidou, F., Zaganas, I., Papastefanakis, E.,Kasselimis, D., Nidos, A., & Simos, P. G. (2014).Age-related decline in verbal learning is moderatedby demographic factors, working memory capacity,and presence of amnestic mild cognitive impairment.Journal of the International NeuropsychologicalSociety, 20, 822–835.

Crossen, J. R., & Wiens, A. N. (1994). Comparison ofthe auditory–verbal learning test (AVLT) andCalifornia Verbal Learning Test (CVLT) in a sampleof normal subjects. Journal of Clinical andExperimental Neuropsychology, 16, 190–194.

Economou, A. (2003). Olfactory identification in elderlyGreek people in relation to memory and attentionmeasures. Archives of Gerontology and Geriatrics, 37,119–130.

Estevez-Gonzalez, A., Kulisevsky, J., Boltes, A., Otermin,P., & Garcia-Sancez, C. (2003). Rey verbal learningtest is a useful tool for differential diagnosis in thepreclinical phase of Alzheimer’s disease: Comparisonwith mild cognitive impairment and normal aging.International Journal of Geriatric Psychiatry, 18,1021–1028.

Ferreira Correia, A., & Campagna Osorio, I. (2014). TheRey Auditory Verbal Learning Test: Normative datadeveloped for the Venezuelan population. Archives ofClinical Neuropsychology, 29, 206–215.

Folia, V., & Kosmidis, M. H. (2003). Assessment ofmemory skills in illiterates: Strategy differences ortest artifacts? Clinical Neuropsychology, 17, 143–152.

Forrester, G., & Geffen, G. (1991). Performance mea-sure of 7-to 15-year-old children on the AuditoryVerbal learning Test. The Clinical Neuropsychologist,5, 345–359.

Fountoulakis, K. N, Tsolaki, M., Chantzi, H., & Kazis, A.(2000). Mini Mental State Examination (MMSE): Avalidation study in Greece. American Journal ofAlzheimer’s Disease and Other Dementias, 15, 342–345.

14 L. MESSINIS ET AL.

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

Fountoulakis, K. N., Tsolaki, M., Iacovides, A.,Yesavage, J., O’Hara, R., Kazis, A., & Ierodiakonou,C. (1999). The validation of the short form of theGeriatric Depression Scale (GDS) in Greece. Aging(Milano),11, 367–372.Geffen, G., Moar, K.P., Clark,C. (1990) .Kosmidis, M.

Gainotti, G., Marra, C., & Villa, G. (2001). A doubledissociation between accuracy and time of executionon attentional tests in Alzheimer’s disease and multi-infarct dementia. Brain, 124, 731–738.

Geffen, G., Moar, K. J., O’Hanlon, A. P., Clark, C. R., &Geffen, L. N. (1990).Performance measures of 16–86-year-old males and females on the Auditory VerbalLearning Test. The Clinical Neuropsychologist, 4, 45–63.

Giannakou, M., & Kosmidis, M. H. (2006). Culturalappropriateness of the Hooper Visual OrganizationTest? Greek normative data. Journal of Clinical andExperimental Neuropsychology, 28, 1023–1029.

Gunther, T., Holtkampa, K., Jolles, J., Herpetz-Dahlamanna, B., & Konrad, K. (2004). Verbal mem-ory and aspects of attentional control in children andadolescents with anxiety disorders or depressive dis-orders. Journal of Affective Disorders, 82, 265–269.

Harris, M. E., Ivnik, R. J., & Smith, G. E. (2002). Mayo’sOlder Americans normative studies: Expanded AVLTrecognition trial norms for ages 57–98. Journal ofClinical and Experimental Neuropsychology, 24, 107–114.

Hatzigeorgiu, H., Gavrilidou, M., Piperidis, S.,Carayannis, G., Papakostopoulou, A.,Spiliotopoulou, A.,. . . E., Iason, D. (2000). Designand implementation of the online ILSP GreekCorpus. Proceedings of the Second InternationalConference on Language Resources and Evaluation(Vol. 3, pp. 1737–1740). Athens: Institute forLanguage and Speech Processing.

Konstantinopoulou, E., Kosmidis, M. H., Ioannidis, P.,Kiosseoglou, G., Karacostas, D., & Taskos, N. (2011).Adaptation of Addenbrooke’s CognitiveExamination–Revised for the Greek population.European Journal of Neurology, 18, 442–447.

Kosmidis, M. H., Vlahou, C. H., Panagiotaki, P., &Kiosseoglou, G. (2004). The verbal fluency task inthe Greek population: Normative data and clusteringand switching strategies. Journal of the InternationalNeuropsychological Society, 10, 164–172.

Lannoo, E., & Vingerhoets, G. (1997). Flemish norma-tive data on common neuropsychological tests:Influence of age, education, and gender. PsychologicaBelgica, 37, 141–155.

Lee, T. M. C. (2003). Normative data:Neuropsychological measures for Hong Kong Chinese.Hong Kong: The University of Hong-Kong,Neuropsychology Laboratory.

Lezak, M. D., Lezak, M.D., Howieson, D. B., (2004). B.,Bigler, E. D., & Tranel, D. (2012). Neuropsychologicalassessment (5th ed.). New York, NY: OxfordUniversity Press

Lezak, M. D., Howieson, D. B., & Loring, D. W. (2004).Neuropsychological assessment (4th ed.). New York,NY: Oxford University Press.

Malec, J. F, & Thompson, J. M. (1994). Relationship ofthe Mayo-Portland adaptability inventory to func-tional outcome and cognitive performance measures.Journal of Head Trauma Rehabilitation, 9, 1–15.

McKhann, G., Drachman, D., Folstein, M., Katzman, R.,Price, D., & Stadlan, E. M. (1984). Clinical diagnosisof Alzheimer’s disease: Report of the NINCDS-ADRDA Work Group under the auspices ofDepartment of Health and Human Services TaskForce on Alzheimer’s Disease. Neurology, 34, 939–944.

Messinis, L., Lada, E., Famellou, P., Chrisoulaki, K.,Konstantinos, K., & Papathanasopoulos, P. (2006,March). Normative data for the WCST-64 card editionin Greece. Paper presented at the 20th PanhellenicCongress of Neurology, Larissa.

Messinis, L., Lyros, E., Georgiou, V., &Papathanasopoulos, P. (2009). Benton VisualRetention Test performance in normal adults andacute stroke patients: Demographic considerations,discriminant validity and test-retest reliability. TheClinical Neuropsychologist, 23, 962–971.

Messinis, L., Malegiannaki, A., Christodoulou., T., &Papathanasopoulos, P. (2011). Color Trails Test:Normative data and criterion validity for the Greekadult population. Archives of Clinical Neuropsychology,26, 322–330.

Messinis, L., Tsakona, I., Malefaki, S., &Papathanasopoulos, P. (2007).Normative data anddiscriminant validity of Rey’s Auditory VerbalLearning Test for the Greek adult population.Archives of Clinical Neuropsychology, 22, 739–752.

Messinis, L., Tsakona, I., & Papathanasopoulos, P.(2006, July). Rey Auditory Verbal Learning Test(RAVLT): Normative data for the Greek adult popula-tion. Poster presented at the InternationalNeuropsychological Society 34th mid-year meeting,Zurich.

Miatton, M., Wolters, M., Lannoo, E., & Vingerhoets, G.(2004). Updated and extended normative data ofcommonly used neuropsychological tests.Psychologica Belgica, 44, 189–216.

Mitchell, A. J., & Shiri-Feshki, M. (2009). Rate of pro-gression of mild cognitive impairment to dementia:Meta-analysis of 41 robust inception cohort studies.Acta Psychiatrica Scandinavica, 119, 252–265.

Mitrushina, M. N., Boone, K. B., Razani, J., & D’Elia, L.F. (2005). Handbook of normative data for neurop-sychological assessment (2nd ed.). New York, NY:Oxford University Press.

Mitrushina, M., Satz, P., Chervinsky, A., & D’Elia, L.(1991). Performance of four age groups of elderly onthe Rey Auditory-Verbal Learning Test. Journal ofClinical Psychology, 47, 351–357.

MitrushinaM.N., Boone, K. (2005). Handbook of norma-tive data for neuropsychological assessment (2nd ed.).

National Statistical Office of Greece. (2014). (Vol. 698,p. 2). Retrieved from http://www.statistics.gr/portal/page/portal/ESYE/BUCKET/General/FEK_monimos_rev.pdf

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY 15

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

Nordlund, A., Rolstad, S., Klang, O., Lind, K., Hansen,S., & Wallin, A. (2007). Cognitive profiles of mildcognitive impairment with and without vascular dis-ease. Neuropsychology, 21, 706–712.

Papathanasiou, A., Messinis, L., Georgiou, L. V., &Papathanasopoulos, P. (2014). Cognitive impairmentin relapsing remitting and secondary progressivemultiple sclerosis patients: Efficacy of a computerizedcognitive screening battery. ISRN Neurology, ArticleID 151379, 7 pp. doi:10.1155/2014/151379

Peruzza Marchianni, N. C., Figuerdo Balthazar, M. L.,Cendes, F., & Pereira Damasceno, B. (2008).Hippocampal atrophy and verbal episodic memoryperformance in amnestic mild cognitive impairmentand mild Alzheimer’s disease: A preliminary study.Dementia and Neuropsychologia, 2, 37–41.

Petersen, R. C., Doody, R., Kurz, A., Mohs, R. C.,Morris, J. C., Rabins, P. V.,. . . Winblad, B. (2001).Current concepts in mild cognitive impairment.Archives of Neurology, 58, 1982–1985.

Petersen, R. C., Smith, G. E., Waring, S. C., Ivnik, R. J.,Tangalos, E. G., & Kokmen, E (1999). Mild cognitiveimpairment: Clinical characterization and outcome.Archives of Neurology, 56, 303–308.

Petersen, R. C., Stevens, J. C., Ganguli, M., Tangalos,E. G., Cummings, J. L., & Dekosky, S. T. (2001).Practise parameter: Early detection of dementia:Mild cognitive impairment (an evidence basedreview). Report of the Quality StandardsSubcommitee of the American Academy ofNeurology. Neurology, 56, 1133–1142.

Powell, J. B., & Cripe, L. I. (1991). Assessment of brainimpairment with the Rey Auditory Verbal LearningTest: A comparison with other neuropsychological mea-sures. Archives of Clinical Neuropsychology, 6, 241–249.

Raz, N. (2000). Aging of the brain and its impact oncognitive performance: An integration of structuraland functional findings. In F. Craik & T. Salthouse(Eds.), The handbook of aging and cognition (2nd ed.,pp. 1–90). Mahwah, NJ: Lawrence Erlbaum Associates.

Rey, A. (1964). L’examen clinique en psychologie[Clinical tests in psychology]. Paris: PressesUniversitaires de France.

Ricci, M., Graef, S., Blundo, C., & Miller, L. A (2012).Using the Rey Auditory Verbal Learning Test(RAVLT) to differentiate Alzheimer’s dementia andbehavioural variant fronto-temporal dementia. TheClinical Neuropsychologist, 26, 926–941.

Savage, R. M., & Gouvier, W. D. (1992). Rey Auditory-Verbal Learning Test: The effects of age and gender,and norms for delayed recall and story recognitiontrials.Archives of Clinical Neuropsychology, 7, 407–414.

Schmidt, M. (1996). Rey Auditory Verbal Learning Test:A handbook. Los Angeles, CA: Western PsychologicalServices.

Schoenberg, M. R., Dawson, K. A., Duff, K., Patton, D.,Scott, J. G., & Adams, R. L. (2006). Test perfor-mance and classification statistics for the ReyAuditory Verbal Learning Test in selected clinicalsamples. Archives of Clinical Neuropsychology, 21,693–703.

Siraly, E., Szabo, A., Szita, B., Kovacs, V., Fodor, Z.,Marosi, C.,. . . Csukly, G. (2015). Monitoring theearly signs of cognitive decline in elderly by computergames: An MRI study. PLoS ONE, 10(2), e0117918.

Speer, P., Wersching, H., Bruchmann, S., Bracht, D.,Stehling, C., Thielsch, M.,. . . Lohmann, H. (2014).Age-and gender adjusted normative data of Rey’sAuditory Verbal Learning Test from healthy subjectsaged between 50 and 70 years. Journal of Clinical andExperimental Neuropsychology, 36, 32–42.

Steinberg, B. A., Bieliauskas, L. A, Smith, G. E., Ivnik, R. J., &Malec, J. F. (2005). Mayo’s older Americans normativestudies: Age and IQ-adjusted norms for the AuditoryVerbal Learning Test and the Visual Spatial LearningTest. The Clinical Neuropsychologist, 19, 464–523.

Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). Acompendium of neuropsychological tests: Administration,norms, and commentary (3rd ed.). New York, NY:Oxford University Press.

Summers, M. J., & Saunders, N. L. J. (2012).Neuropsychological measures predict decline toAlzheimer’s dementia from mild cognitive impair-ment. Neuropsychology, 26, 498–508.

Tierney, M. C., Nores, A., Snow, W. G., Fisher, R. H.,Zorzitto, M. L., & Reid, D. W. (1994). Use of the ReyAuditory Verbal Learning Test in differentiating nor-mal aging from Alzheimer’s and Parkinson’s demen-tia. Psychological Assessment, 6, 129–134.

Uchiyama, C. L., D’Elia, L. F., Dellinger, A. M., Becker,J. T., Selnes, O. A., Welsch, J. E.,. . . Miller, E. N.(1995). Alternate forms of the auditory-verbal learn-ing test: Issues of test comparability, longitudinalreliability, and moderating demographic variables.Archives of Clinical Neuropsychology, 10, 133–145.

Vakil, E., & Blachstein, H. (1997). Rey AVLT:Developmental norms for adults and the sensitivityof different memory measures to age. The ClinicalNeuropsychologist, 11, 356–369.

Vakil, E., Greenstein, Y., & Blachstein, H. (2010).Normative data for composite scores for childrenand adults derived from the rey auditory verbal learn-ing test. The Clinical Neuropsychologist, 24, 662–677.

Van der Elst, W., Van Boxtel, M. P. J., Van Breukelen,G. J. P., & Jolles, J. (2005). Rey’s verbal learning test:Normative data for 1855 healthy participants aged24–81 and the influence of age, sex, education, andmode of presentation. Journal of the InternationalNeuropsychological Society, 11, 290–302.

Vlahou, C. H., & Kosmidis, M. H. (2002). The Greek TrailMaking Test: Preliminary normative data for clinicaland research use [in Greek]. Psychology: The Journal ofthe Hellenic Psychological Society, 9, 336–352.

Vlahou, C. H., Kosmidis, M. H., Dardagani, A., Tsotsi, S.,Giannakou, M., Giazkoulidou, A.,. . . Pontikakis, N.(2013). Development of the Greek Verbal LearningTest: Reliability, construct validity, and normative stan-dards. Archives of Clinical Neuropsychology, 28, 52–64.

Wiens, A. N., McMinn, M. R., & Crossen, J. R. (1988).Rey Auditory–Verbal Learning Test: Development ofnorms for healthy young adults. The ClinicalNeuropsychologist, 2, 67–87.

16 L. MESSINIS ET AL.

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015

Zalonis, I., Kararizou, E., Christidi, F., Kapaki, E.,Triantafyllou, N. I., Varsou, A.,. . . Vassilopoulos, D.(2009). Selective Reminding Test: Demographic pre-dictors of performance and normative data for theGreek population. Psychological Report, 104, 593–607.

Zeidman, Y., Schweiger, A., Korczyn, A. D., Vachapov,V., Verhovsky, R., Halperin, I.,. . . Andelman, F.

(2008). Rey Auditory Verbal Learning Test as a toolfor diagnosis of mild cognitive impairment. ActaNeuropsychologica, 6, 169–176.

Zygouris, S., & Tsolaki, M. (2015). Computerized cog-nitive testing for older adults: A review. AmericanJournal of Alzheimer’s Disease and Other Dementias,30, 13–28.

JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY 17

Dow

nloa

ded

by [

Uni

vers

ity o

f Pa

tras

], [

Dr

Lam

bros

Mes

sini

s] a

t 01:

47 2

4 N

ovem

ber

2015