are cognitive screening tools sensitive and specific ... · keywords included stroke, cerebrovasc*,...

3129

It is estimated that up to three quarters of acute and subacute stroke survivors exhibit cognitive impairment, with many

experiencing ongoing problems.1,2 Cognitive impairment can significantly compromise functional recovery, quality of life, and social engagement after stroke.2–4 Encouragingly early detection and rehabilitation can improve functional recovery of stroke-related impairments.5 Unfortunately, however, a sig-nificant amount of cognitive dysfunction is not detected by health professionals in acute and subacute settings.6

Comprehensive neuropsychological assessment using reli-able and valid tools to measure multiple cognitive domains is considered the gold standard method of detecting and characterizing cognitive dysfunction after stroke. However, neuropsychological assessments are often considered too expensive and lengthy to be routinely administered to patients with stroke. In an attempt to improve detection of cognitive impairments, while managing expense, many national stroke clinical management guidelines now recommend the use of screening measures to detect cognitive impairment.7–9 If cog-nitive difficulties are detected during this screening process, comprehensive assessment and intervention is then recom-mended. The Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA) are 2 screening tools that are regularly used in clinical practice. Although these tests are commonly used to detect cognitive impairment in dementia settings, neither was specifically designed for use after stroke. The profile of cognitive impairment after stroke is heterogeneous, and focal impairments such as dysphasia, dys-praxia, unilateral inattention, and agnosia are often observed. Therefore, we cannot assume that reliability and validity of cognitive screening tools found in other clinical populations will be comparable in stroke.

It is acknowledged that numerous reliability and validity indices are important to consider when evaluating neuropsy-chological measures. However, when considering the use of cognitive screening measures, sensitivity, specificity, positive

predictive value (PPV), and negative predictive value (NPV) are particularly important to ensure patients with cognitive impairment are not missed, and patients without cognitive impairment do not undergo comprehensive neuropsycho-logical evaluation unnecessarily. Several studies have inves-tigated the sensitivity and specificity of cognitive screening tools within stroke populations. However, a range of differ-ent methodologies have been used, and results seem to vary considerably across studies. Thus, the aims of this review were (1) to systematically review the sensitivity, specificity, PPV, and NPV of a range of cognitive screening tools used in stroke and (2) to critically evaluate methodologies used within these studies. It is intended that findings from this review will inform clinicians regarding suitability of these screening tools for clinical use and direct best practice for future research in this field.

MethodsSearch StrategyThis systematic literature review was conducted and reported in line with the current Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Articles were identified through MedLine, PsychInfo, Scopus, PubMed, and CINAHL da-tabases. Keywords included stroke, cerebrovasc*, cognit*, screen*, sensitivity, and specificity. Common screening measure names were also used. See Figure I in the online-only Data Supplement for an example of key words and search strategy. The search was limited to studies of adult humans published in English. The electronic search was conducted on December 27, 2013. Reference lists of articles in-cluded in this review and other relevant publications were also used to identify any studies overlooked in the electronic search.

Study SelectionArticles were included in this review if they met 3 key criteria: (1) male or female participants aged ≥18 years; (2) confirmed ischemic or hemorrhagic stroke, and (3) analysis of the sensitivity and specificity of a cognitive screening measure compared with a gold standard neu-ropsychological assessment. If >1 clinical population was included

Are Cognitive Screening Tools Sensitive and Specific Enough for Use After Stroke?

A Systematic Literature Review

Renerus J. Stolwyk, DPsych; Megan H. O’Neill, BPsych(Hons); Adam J.D. McKay, PhD; Dana K. Wong, PhD

Received February 18, 2014; final revision received May 27, 2014; accepted July 1, 2014.From the School of Psychological Sciences, Monash University, Melbourne, Australia (R.J.S., M.H.O., A.J.D.M., D.K.W.); and Epworth Rehabilitation,

Melbourne, Australia (A.J.D.M.).The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.114.004232/-/DC1.Correspondence to Renerus J. Stolwyk, DPsych, School of Psychological Sciences, Building 17, Clayton Campus, Monash University, Melbourne,

Victoria, 3800, Australia. E-mail [email protected](Stroke. 2014;45:3129-3134.)© 2014 American Heart Association, Inc.

Stroke is available at http://stroke.ahajournals.org DOI: 10.1161/STROKEAHA.114.004232

Section Editors: Amanda Thrift, PhD, and Barbara G. Vickrey, MD, MPH

Topical Review

by guest on August 5, 2017

http://stroke.ahajournals.org/D

ownloaded from

by guest on A

ugust 5, 2017http://stroke.ahajournals.org/

Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

by guest on A


Dow

nloaded from



ownloaded from

http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.114.004232/-/DC1

mailto:[email protected]

http://stroke.ahajournals.org/

























3130 Stroke October 2014

in a study (eg, transient ischemic attack and stroke), stroke-specific data must have been available. Cognitive screening measures were included if they were designed to screen for cognitive impairment or had been used for that purpose and typically took <30 minutes to administer. Gold standard neuropsychological assessments were included if they used multiple domain-specific neuropsychological assessments with established reliability and validity.10 Some studies identified during the literature search investigated screening tools that aim to detect just 1 cognitive domain, such as dysphasia or dyspraxia. However, understandably these studies typically only included 1 cog-nitive domain within their gold standard assessment and thus did not meet our eligibility criteria.

In line with PRISMA guidelines, 2 authors (R.J.S. and M.H.O.) separately reviewed results from the electronic search and identified potentially relevant titles and abstracts. If the abstract suggested the article met the inclusion criteria, the full-text article was obtained and evaluated. Full-text articles were then compared across authors, and contrasting/ambiguous studies were discussed to determine whether they met criteria for inclusion. Articles that met the inclusion criteria were included for subsequent data extraction.

Data ExtractionThe following data were extracted from each article: author, year, title, participant data (sample size, age, sex, education), stroke data (mechanism, location, hemisphere, severity), recruitment procedures (inclusion/exclusion criteria, participant attrition), cognitive screen-ing measure used, domains and tests included in gold standard cog-nitive assessment, time poststroke of screening and gold standard cognitive assessments, and sensitivity, specificity, PPV, and NPV results. In some studies, multiple sets of sensitivity, specificity, PPV, and NPV data were presented at different screening measure cut points. To limit the amount of data presented, the cut point that re-sulted in the most favorable sensitivity and specificity results was se-lected. This was based on commonly used criteria of sensitivity >80% and specificity >60%.11,12

ResultsSearch ResultsElectronic and additional searching returned 13 201 records; duplicate ones were removed. Sixty-six records remained following title and abstract screening. A further 50 records were excluded during full-text review for the following rea-sons: review articles (n=2), combined transient ischemic attack/stroke data (n=3), full neuropsychological battery used instead of screening tool (n=4), screening measure did not meet inclusion criteria (n=3), not written in English (n=2), no sensitivity and specificity data (n=6), nonstroke samples used (n=11), and gold standard neuropsychological battery did not meet inclusion criteria (n=19). Sixteen articles were found to meet our inclusion criteria and were retained for analysis. See Figure II in the online-only Data Supplement for a summary of the above.

Study ParticipantsSummary of study descriptions and evaluations is presented in Tables I and II in the online-only Data Supplement. All stud-ies adequately reported sample size; however, few justified the sample size used or reported whether assumptions for statis-tical analyses were met. Based on key references within this field,13,14 it seems that many studies did not use sufficient sample sizes, particularly those that used samples of ≤50 people.15–18 This may have contributed to the large confidence intervals of sensitivity and specificity results noted across studies.

With regard to demographic variables, all studies pro-vided adequate information regarding age and sex. Four studies failed to include education information.11,15,16,19 Most study samples appeared representative of stroke populations. However, most studies excluded nonnative language speak-ers. With regard to stroke variables, most studies reported key information such as stroke mechanism (12 of 16), location (12 of 16), and severity (11 of 16). Stroke mechanism and lesion location variables across studies appeared generally represen-tative of stroke populations. However, most studies excluded people with severe stroke. Only 2 studies provided adequate statistical analysis of whether the final study sample used for analysis was representative of the wider patient group within their clinical setting.20,21 There was significant heterogeneity across studies regarding time since stroke, with mean/median times ranging from 6 days17 to 29 months18 across studies. Two studies failed to report this information,15,22 and others provided only limited information.

Sensitivity and Specificity MethodologyAll studies calculated sensitivity and specificity using receiver operative characteristics curve analysis. Only 9 of 16 stud-ies calculated PPV or NPV data. Gold standard assessments used to classify the cognitive status of participants differed across studies. Cognitive domains such as language, visual/space perception, attention, memory, and executive func-tions were generally well represented. However, cognitive functions such as calculic function, praxis, and mental speed were less well represented, included in ≤4 of the 16 studies. Most studies used age- and education-based normative data to interpret gold standard test performances, using a crite-rion ranging from fifth to tenth percentile as an indicator of impaired performance. However, studies differed regarding whether impairment on single or multiple cognitive domains was required to classify participants as impaired on gold stan-dard assessments. Furthermore, some studies did not use psy-chometric criteria at all, instead relying on clinician opinion taken from neuropsychological reports.11,15,20 To reliably and validly assess sensitivity and specificity, it was expected that screening and gold standard assessments would be conducted within a short period of each other. Unfortunately, 5 studies did not sufficiently report this information,16,18,20,23,24 and 3 reported the mean time interval between assessments as >10 days.11,15,25 Only 6 studies stratified sensitivity or specificity results according to demographic or stroke variables.20,22,23,25–27

Sensitivity and Specificity ResultsAs seen in Table, the MMSE and MoCA were the most com-monly studied screening measures. With regard to the MMSE, 11 studies investigated the sensitivity and specificity of the mea-sure, with just 3 reporting sensitivity and specificity at or above commonly regarded acceptable levels (sensitivity >80% and specificity >60%).23,25,27 All 3 studies obtained these levels using cut points of either 26/30 or 27/30. Bour et al25 achieved accept-able sensitivity and specificity levels only when the gold standard assessment impairment criterion was increased to ≥2 cognitive domains. Of the 3 studies that reported adequate sensitivity and specificity, PPVs were generally >80%; however, NPVs were less impressive, ranging from 65% to 73% across studies.



ownloaded from


Stolwyk et al Cognitive Screening After Stroke 3131

Table. Results From Included Studies

Study Screening Measure Screening Measure Cut Point SE, % (CI) SP, % (CI) PPV, % (CI) NPV, % (CI)

Agrell et al20 MMSE 23/24 56 80

Blake et al11 MMSE <24 62 88

Boosman et al26 BNI Age <55, <47 80 (49–94) 39 (20–61) 61 93

Age >55, <41 92 (67–99)* 84 (58–96)* 92 96

Bour et al25 MMSE 27/28 ≥1 impaired gold standard domain

72 71 93

27/28 ≥2 impaired gold standard domains

80* 70* 86

26/27 ≥4 impaired gold standard domains

82* 75* 72

Cartoni et al15 MEAMS ≥3 failed subtests 52 (32–71) 100 (29–100)

≥5 failed subtests 26 (11–46) 100 (29–100)

Cumming et al27 MMSE 26/27† 82* 76* 86 70

MoCA

23/24 92* 67* 84 82

Godefroy et al23 MMSE ≤26 80* 77* 88 65

MoCA ≤23 88* 71* 86 73

MMSEadj ≤27 89* 61* 83 73

MoCAadj

(adjusted scores created using control group to account for age and education)

≤23 84* 81* 90 71

Grace et al22 MMSE <27 81 45

Modified MMSE <86 94 50

Green et al28 RBANS 83/84 84* 90* 98 53

Morris et al12 ACE-R 82 80 40 87 28

MMSE 27 80 20 84 16

Nøkleby et al16 Cognistat 8/9 81 (68–93)* 67 (22–96)*

SINS 2/3 71 (57–85) 67 (22–96)

CDT 9/10 63 (49–78) 67 (22–96)

Nys et al17 MMSE <27 96 40

Salvadori et al21 MoCA 21 91* 76* 80 89

Schweizer et al18 MMSE <27

MoCA <26 Domain-specific data provided. No global sensitivity/

specificity data; MMSE: sensitivity=0 for all cognitive domains, specificity=1.00 for all cognitive domains; MoCA: sensitivity ranged from 40 to

100 across domains; specificity ranged from 54 to 70 across

domains

Srikanth et al24 MMSE-S ≤23 14 (4–32) 100 (92–100) 100 (40–100) 63 (50–74)

Wong et al19 MoCA 2–4 wk 17/18 75 (43–95) 95 (87–99) 75 (41–95) 95 (87–99)

1 y 21/22 100 (74–100)* 75 (63–85)* 41 (24–61) 100 (93–100)

MMSE 2–4 wk 23/24 75 (43–95) 90 (80–96) 60 (32–84) 95 (86–99)

1 y 23/24 58 (28–85) 84 (73–92) 39 (17–64) 92 (82–97)

ACE-R indicates Addenbrooke Cognitive Examination, Revised; AUC, area under the receiver operating curve; BNI, Barrow Neurological Institute (screen for higher cerebral functions); CDT, clock drawing task; CI, confidence interval; MEAMS, Middlesex Elderly Assessment of Mental State; MMSE, Mini-Mental State Examination; MMSE-S, MMSE-Standardized; MoCA, Montreal Cognitive Assessment; NPV, negative predictive value; PPV, positive predictive value; RBANS, Repeatable Battery for the Assessment of Neuropsychological Status; SE, sensitivity; SINS, Screening Instrument for Neuropsychological Impairments in Stroke; and SP, specificity.

*Results reach traditionally acceptable levels of 80% SE and 60% SP.†Improved AUC for MMSE noted when z score criterion increased to −1.5.



ownloaded from



Five studies investigated the sensitivity and specificity of the MoCA. Three of these reported adequate sensitivity and specificity21,23,27; 1 further study reported adequate sensitiv-ity and specificity at 1 year poststroke, but not 2 to 4 weeks poststroke,19 and the final study reported adequate sensitiv-ity and specificity for only 2 of the 13 gold standard tests included (naming and verbal learning).18 Acceptable sensitiv-ity and specificity were found at different MoCA cut points across the studies, ranging from 21 to 26. Of the 4 studies that reported adequate sensitivity and specificity, only 2 studies also reported PPVs and NPVs >80%.21,27

Four studies directly compared the MMSE and MoCA. Using area under the receiver operating curve scores, 2 stud-ies reported no significant differences between the MMSE and MoCA,23,27 whereas another reported higher MoCA area under the receiver operating curve scores compared with the MMSE at 1 year poststroke only.19 The final study reported superior sensitivity of the MoCA compared with the MMSE.18 The general trend noted across these 4 studies was of somewhat better sensitivity and slightly poorer specificity of MoCA compared with the MMSE.

The sensitivity and specificity of other screening measures have been investigated in only a single study that met our inclu-sion criteria. Both the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) and Cognistat dem-onstrated typically accepted sensitivity and specificity levels; however, NPV was only 53% for the RBANS, and no PPV or NPV data were presented for the Cognistat.16,28 The Barrow Neurological Institute (BNI) screen for higher cerebral func-tions returned acceptable sensitivity, specificity, PPV, and NPV levels in people aged >55 years but not <55 years.26 The Middlesex Elderly Assessment of Mental State (MEAMS), Addenbrooke Cognitive Examination, Revised (ACE-R), Screening Instrument for Neuropsychological Impairments in Stroke (SINS), and Clock Drawing Test all failed to achieve adequate levels of sensitivity and specificity.12,15,16

In addition to providing total scores, some cognitive screen-ing measures also provided domain-specific subscores. Four studies included in this review examined the sensitivity and specificity of these subscores to detect domain-specific cogni-tive impairment assessed using gold standard measures.12,15,16,28 This information is presented in Table III in the online-only Data Supplement. Three of the 5 subscores from the RBANS achieved acceptable sensitivity and specificity levels (immedi-ate memory, language, visuospatial) with the other 2 subtests only just under acceptable thresholds (delayed memory, atten-tion).28 Results from the Cognistat, MEAMS, ACE-R, and SINS were less impressive. The Cognistat and MEAMS both achieved acceptable results for their naming subscores; how-ever, all other subscores did not reach acceptable levels.15,16 No subscores from ACE-R or SINS reached acceptable lev-els.12,16 Although memory, language, and visuospatial domains were regularly assessed, attention, processing speed, praxis, and executive function were seldom examined.

Few studies specifically investigated whether method-ological factors or patient variables impacted on sensitivity and specificity results. Qualitatively there was no consistent evidence to suggest time poststroke significantly affected

sensitivity and specificity results. Although Wong et al19 reported acceptable MoCA sensitivity and specificity at 1 year but not 2 to 4 weeks after stroke, other studies reported favorable MoCA, Cognistat, RBANS, and BNI sensitivity and specificity ranging from 1 week to 1 year poststroke. With regard to time interval between screening and gold standard assessments, almost all studies that reported favorable sensi-tivity and specificity of the MoCA, Cognistat, RBANS, and BNI used mean assessment time intervals within ≈1 week. Studies varied regarding their criteria for impairment on gold standard assessment. Those who used a multiple cognitive domain criterion were more likely to report adequate sensitiv-ity and specificity compared with those who used a single-domain criterion. A minority of studies stratified sensitivity and specificity results according to demographic and stroke variables. One study reported better sensitivity and specific-ity results in older stroke participants.26 Lesion hemisphere effects were equivocal. One study reported no effect,25 whereas others reported better results in right hemisphere20,27 or left hemisphere groups.22 No studies specifically investigated the impact of premorbid cognitive function, stroke severity or mechanism, or cultural factors on screening measure perfor-mance. Few studies directly compared screening measure per-formance across different gold standard cognitive domains. The MoCA was shown to be relatively more sensitive to nam-ing and verbal learning difficulties compared with other cog-nitive domains in 1 study,18 whereas performance was higher for language and visuospatial impairments in another.19

DiscussionSixteen articles that investigated the sensitivity and specificity of cognitive screening tools in stroke met our inclusion crite-ria. Eleven of these studies investigated the MMSE, and most reported inadequate sensitivity and specificity. MoCA results were somewhat better, with 3 of 5 studies reporting consistent acceptable sensitivity and specificity results. It is not clear why the MoCA performed better than the MMSE. However, pos-sible reasons include the fact the MoCA contains items assess-ing executive functions, which are often affected by stroke and the total score of MoCA can be adjusted for education level, albeit crudely. Interestingly, 2 relatively more recently devel-oped measures, the RBANS and Cognistat, demonstrated tra-ditionally acceptable levels of sensitivity and specificity. There is also some preliminary support for the use of BNI within older stroke populations. Furthermore, analysis of RBANS subscores highlighted promising sensitivity and specificity results to detect a range of focal cognitive difficulties, includ-ing memory, language, and visuospatial difficulties. However, it is noted that the RBANS, Cognistat, and BNI were only investigated in 1 study that met our eligibility criteria, and fur-ther research confirming these initial findings is warranted.

The above findings provide some preliminary support for the use of the MoCA, BNI, Cognistat, and RBANS as screening measures for stroke. However, these findings should be consid-ered in the context of some key methodological issues. First, of the 7 studies that reported adequate sensitivity and specific-ity of MoCA, BNI, Cognistat, and RBANS, 3 either failed to report PPVs and NPVs16 or reported NPVs <80% (indicating



ownloaded from


Stolwyk et al Cognitive Screening After Stroke 3133

≥20% false-negative rates).23,28 Second, adequate sensitiv-ity and specificity of the MoCA were found at different cut points, making recommendations for clinical practice difficult. Third, most studies did not include calculation, praxis, and speed of information processing within gold standard assess-ments. Thus, capacity for screening measures to detect these cognitive difficulties remains unknown, which is problematic considering that impairments of calculic, praxis, and mental speed functions are not uncommon after stroke and can signifi-cantly impact functional recovery.1 Fourth, most studies that reported adequate sensitivity and specificity used a criterion of multiple cognitive impairments (≥2 domains) within their gold standard assessments. Studies have shown higher screening measure sensitivity for multiple-domain versus single-domain cognitive impairments.25,29 Thus sensitivity results from stud-ies that used multiple-domain impairment as a gold standard criterion may have been lower if a single-domain criterion was used (although equally specificity results may have been higher). Finally, few studies stratified sensitivity and specific-ity results according to demographic and stroke variables. This can be problematic for several reasons. For example, many screening measures do not account for age, education, or pre-morbid intelligence. Thus, it is possible that sensitivity of these screening measures for young and highly intelligent people may be limited, and specificity may be limited in older people and those with lower premorbid intelligence. Furthermore, people with severe stroke and those from culturally and lin-guistically diverse backgrounds were often excluded from studies altogether. Additional research is required within these groups before use of these screening measures is warranted. See Figure for recommendations for future research.

With regard to more general methodological issues, signifi-cant heterogeneity and poor reporting regarding time interval between screening and gold standard assessments and time of assessment since stroke were noted across studies. Unless long-term predictive validity is a specific research aim, we recommend screening and gold standard assessments be con-ducted as time-congruent as possible. Cognitive function can change significantly during the course of stroke recovery, and results from early screening cannot be assumed to be

an accurate picture of longer-term cognitive function. Thus, we recommend further research investigating the potential impact of time since stroke on the sensitivity and specificity of screening measures. On another note, although it is important to report PPV and NPV data, we acknowledge these values vary according to prevalence of impairment in the population. Thus, direct comparison of these values across studies is not valid. Importantly, however, PPVs and NPVs can be calcu-lated based on sensitivity, specificity, and prevalence data.30 Thus, clinicians and researchers alike may choose to use data presented in this review to estimate PPV and NPV across a range of stroke populations where prevalence data are known. Finally, few studies have investigated which specific cognitive domains are more or less likely to be detected by these screen-ing measures. Further research is warranted.

Many researchers have previously suggested that 80% sen-sitivity and 60% specificity of cognitive screening measures is considered adequate for clinical practice. However, the significant negative impact of cognitive impairment in stroke survivors has been consistently demonstrated.2–4 As such, 20% nondetection of patients with cognitive impairment seems unacceptable for clinical practice. Further research is required to more comprehensively examine existing screening mea-sures that show initial promise (MoCA, Cognistat, RBANS, BNI) addressing previous methodological weaknesses noted above. Further development of more appropriate stroke-spe-cific screening measures may be warranted if future research does not generate positive results. Furthermore, it is impor-tant to evaluate how current recommended guidelines (cog-nitive screening followed by comprehensive assessment) are being implemented in clinical practice. There is evidence to suggest good adherence to cognitive screening protocols, but limited provision of further comprehensive assessment when indicated by screening results.31 Further research exploring potential modifications to screening processes is also war-ranted. For example, benefits of including patient, close other, or clinician reports of cognitive difficulties, in conjunction with screening measures, to improve detection of cognitive difficulties could be explored. Addition of items not included in current screening measures, but often affected by stroke such as calculation, praxis, and mental speed, should also be considered. It would be particularly helpful for these cogni-tive measures to be incorporated as standard measures within stroke trials to ensure ongoing comprehensive investigation of their utility across research and clinical settings. Although beyond the scope of this review, it is also important to con-sider cognitive screening in other cerebrovascular disorders. For example, some screening protocols have been specifically developed for small-vessel disease and have demonstrated encouraging results.32 This may be because of the relatively more homogeneous neuropathology and associated cognitive profile seen in this population, compared with the relatively more heterogeneous cognitive profile across the stroke popu-lation, which seems to present as a challenge for some existing screening measures to accommodate.

In conclusion, a limited number of studies have adequately investigated the sensitivity and specificity of cognitive screen-ing measures after stroke. Although most studies do not support the MMSE for clinical use, the MoCA, Cognistat,

Figure. Recommended methodological considerations for future studies. CALD indicates culturally and linguistically diverse; NPV, negative predictive value; PPV, positive predictive value; SE, sen-sitivity; and SP, specificity.



ownloaded from



RBANS, and BNI show some initial promise. However, fur-ther research addressing key methodological considerations and further discussion regarding what is considered accept-able sensitivity and specificity for clinical practice is required before use of these screening measures can be fully supported.

DisclosuresNone.

References 1. Leśniak M, Bak T, Czepiel W, Seniów J, Członkowska A. Frequency

and prognostic value of cognitive disorders in stroke patients. Dement Geriatr Cogn Disord. 2008;26:356–363.

2. Nys GM, van Zandvoort MJ, de Kort PL, van der Worp HB, Jansen BP, Algra A, et al. The prognostic value of domain-specific cognitive abili-ties in acute first-ever stroke. Neurology. 2005;64:821–827.

3. Hochstenbach JB, Anderson PG, van Limbeek J, Mulder TT. Is there a relation between neuropsychologic variables and quality of life after stroke? Arch Phys Med Rehabil. 2001;82:1360–1366.

4. Hommel M, Miguel ST, Naegele B, Gonnet N, Jaillard A. Cognitive deter-minants of social functioning after a first ever mild to moderate stroke at vocational age. J Neurol Neurosurg Psychiatry. 2009;80:876–880.

5. Paolucci S, Antonucci G, Grasso MG, Morelli D, Troisi E, Coiro P, et al. Early versus delayed inpatient stroke rehabilitation: a matched compari-son conducted in Italy. Arch Phys Med Rehabil. 2000;81:695–700.

6. Jaillard A, Naegele B, Trabucco-Miguel S, LeBas JF, Hommel M. Hidden dysfunctioning in subacute stroke. Stroke. 2009;40:2473–2479.

7. Intercollegiate Stroke Working Party. National Clinical Guidelines for Stroke. 4th ed. London, UK: Royal College of Physicians; 2012.

8. National Stroke Foundation. Clinical Guidelines for Stroke Management 2010. Melbourne, Australia: National Stroke Foundation; 2010.

9. Lindsay MP, Gubitz G, Bayley M, Hill MD, Davies-Schinkel C, Singh S, Phillips S. Canadian Best Practice Recommendations for Stroke Care (Update 2010). Ottawa, Canada: Canadian Stroke Network; 2010.

10. Lezak MD, Howieson DB, Bigler ED, Tranel D. Neuropsychological Assessment. 5th ed. New York, NY: Oxford University Press; 2012.

11. Blake H, McKinney M, Treece K, Lee E, Lincoln NB. An evaluation of screening measures for cognitive impairment after stroke. Age Ageing. 2002;31:451–456.

12. Morris K, Hacker V, Lincoln NB. The validity of the Addenbrooke’s Cognitive Examination-Revised (ACE-R) in acute stroke. Disabil Rehabil. 2012;34:189–195.

13. Carley S, Dosman S, Jones SR, Harrison M. Simple nomograms to calcu-late sample size in diagnostic studies. Emerg Med J. 2005;22:180–181.

14. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36.

15. Cartoni A, Lincoln NB. The sensitivity and specificity of the Middlesex Elderly Assessment of Mental State (MEAMS) for detecting cognitive impairment after stroke. Neuropsychol Rehabil. 2005;15:55–67.

16. Nøkleby K, Boland E, Bergersen H, Schanke AK, Farner L, Wagle J, et al. Screening for cognitive deficits after stroke: a comparison of three screening tools. Clin Rehabil. 2008;22:1095–1104.

17. Nys GM, van Zandvoort MJ, de Kort PL, Jansen BP, Kappelle LJ, de Haan EH. Restrictions of the Mini-Mental State Examination in acute stroke. Arch Clin Neuropsychol. 2005;20:623–629.

18. Schweizer TA, Al-Khindi T, Macdonald RL. Mini-Mental State Examination versus Montreal Cognitive Assessment: rapid assessment tools for cognitive and functional outcome after aneurysmal subarach-noid hemorrhage. J Neurol Sci. 2012;316:137–140.

19. Wong GK, Lam SW, Wong A, Ngai K, Poon WS, Mok V. Comparison of montreal cognitive assessment and mini-mental state examination in evaluating cognitive domain deficit following aneurysmal subarachnoid haemorrhage. PLoS One. 2013;8:e59946.

20. Agrell B, Dehlin O. Mini mental state examination in geriatric stroke patients. Validity, differences between subgroups of patients, and relation-ships to somatic and mental variables. Aging (Milano). 2000;12:439–444.

21. Salvadori E, Pasi M, Poggesi A, Chiti G, Inzitari D, Pantoni L. Predictive value of MoCA in the acute phase of stroke on the diagnosis of mid-term cognitive impairment. J Neurol. 2013;260:2220–2227.

22. Grace J, Nadler JD, White DA, Guilmette TJ, Giuliano AJ, Monsch AU, et al. Folstein vs modified Mini-Mental State Examination in geriatric stroke. Stability, validity, and screening utility. Arch Neurol. 1995;52:477–484.

23. Godefroy O, Fickl A, Roussel M, Auribault C, Bugnicourt JM, Lamy C, et al. Is the Montreal Cognitive Assessment superior to the Mini-Mental State Examination to detect poststroke cognitive impairment? A study with neuropsychological evaluation. Stroke. 2011;42:1712–1716.

24. Srikanth V, Thrift AG, Fryer JL, Saling MM, Dewey HM, Sturm JW, et al. The validity of brief screening cognitive instruments in the diag-nosis of cognitive impairment and dementia after first-ever stroke. Int Psychogeriatr. 2006;18:295–305.

25. Bour A, Rasquin S, Boreas A, Limburg M, Verhey F. How predictive is the MMSE for cognitive performance after stroke? J Neurol. 2010;257:630–637.

26. Boosman H, Visser-Meily JM, Post MW, Duits A, van Heugten CM. Validity of the Barrow Neurological Institute (BNI) screen for higher cerebral functions in stroke patients with good functional outcome. Clin Neuropsychol. 2013;27:667–680.

27. Cumming TB, Churilov L, Linden T, Bernhardt J. Montreal Cognitive Assessment and Mini-Mental State Examination are both valid cognitive tools in stroke. Acta Neurol Scand. 2013;128:122–129.

28. Green S, Sinclair E, Rodger E, Birks E, Lincoln N. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) for post-stroke cognitive impairment screening. Int J Ther Rehabil. 2013;20: 536–541.

29. Pendlebury ST, Mariz J, Bull L, Mehta Z, Rothwell PM. Impact of dif-ferent operational definitions on mild cognitive impairment rate and MMSE and MoCA performance in transient ischaemic attack and stroke. Cerebrovasc Dis. 2013;36:355–362.

30. Smith JE, Winkler RL, Fryback DG. The first positive: computing posi-tive predictive value at the extremes. Ann Intern Med. 2000;132:804–809.

31. McClure JA, Salter K, Foley N, Mahon H, Teasell R. Adherence to Canadian Best Practice Recommendations for Stroke Care: vascular cog-nitive impairment screening and assessment practices in an Ontario inpa-tient stroke rehabilitation facility. Top Stroke Rehabil. 2012;19:141–148.

32. O’Sullivan M, Morris RG, Markus HS. Brief cognitive assessment for patients with cerebral small vessel disease. J Neurol Neurosurg Psychiatry. 2005;76:1140–1145.

KEY WORDS: cognition ◼ neuropsychology ◼ sensitivity and specificity ◼ stroke



ownloaded from


Renerus J. Stolwyk, Megan H. O'Neill, Adam J.D. McKay and Dana K. WongSystematic Literature Review

Are Cognitive Screening Tools Sensitive and Specific Enough for Use After Stroke?: A

Print ISSN: 0039-2499. Online ISSN: 1524-4628 Copyright © 2014 American Heart Association, Inc. All rights reserved.

is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231Stroke doi: 10.1161/STROKEAHA.114.004232

2014;45:3129-3134; originally published online July 29, 2014;Stroke.

http://stroke.ahajournals.org/content/45/10/3129World Wide Web at:

The online version of this article, along with updated information and services, is located on the

http://stroke.ahajournals.org/content/suppl/2014/07/29/STROKEAHA.114.004232.DC1Data Supplement (unedited) at:

http://stroke.ahajournals.org//subscriptions/

is online at: Stroke Information about subscribing to Subscriptions:

http://www.lww.com/reprints Information about reprints can be found online at: Reprints:

document. Permissions and Rights Question and Answer process is available in the

Request Permissions in the middle column of the Web page under Services. Further information about thisOnce the online version of the published article for which permission is being requested is located, click

can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office.Strokein Requests for permissions to reproduce figures, tables, or portions of articles originally publishedPermissions:



ownloaded from

http://stroke.ahajournals.org/content/45/10/3129

http://stroke.ahajournals.org/content/suppl/2014/07/29/STROKEAHA.114.004232.DC1

http://www.ahajournals.org/site/rights/

http://www.lww.com/reprints

http://stroke.ahajournals.org//subscriptions/


1

SUPPLEMENTAL MATERIAL

Are cognitive screening tools sensitive and specific enough for use following stroke? A systematic literature review.

*Renerus J. Stolwyk, DPsych 1

; Megan H. O’Neill, BPsych(Hons) 1; Adam J.D. McKay, PhD

1, 2; Dana K.Wong, PhD

1

1

School of Psychological Sciences, Monash University, Melbourne, Australia 2

Epworth Rehabilitation, Melbourne, Australia

*Corresponding author: Dr Renerus J. Stolwyk, School of Psychological Sciences, Building 17, Clayton Campus, Monash University,

Melbourne, Victoria, 3800, Australia. Phone: +61 3 9902 0099. Fax: +61 3 9905 3948. Email: [email protected]

2

Supplemental Table I. Descriptions of research methodologies

Study Screening

Measure

Sample Size

(included in

statistical

analyses)

Inclusion and

exclusion criteria

Demographic

Information

Stroke Information Time post-

stroke

Agrell et

al.1

MMSE 116 Inclusion: Stroke

patients on geriatric

rehabilitation unit

Exclusion: Previous

diagnosis of

dementia of

cognitive

impairment

Age: Female M=77

SD=7; Male M=76

SD=7

Sex: Male N=65,

Female N=51

Education: Primary

school only N=85,

higher-level schooling

N=31

Mechanism: Infarction

N=84, Haemorrhage

N=10, Unspecified N=22.

Location: NR

Hemisphere: Right N=60,

Left N=39, Infratentorial

N=17

Severity: Barthel Index

M=67, SD=22

2-8 weeks post

stroke

Blake et al.2

MMSE 112 Inclusion: could sit

and co-operate with

an assessment of 30

minutes

Exclusion:

Significant hearing

or sight issues

Age: M=70.8,

SD=12.2

Sex: Male N=64,

Women N=48

Education: NR

Mechanism: NR

Location: NR

Hemisphere: Left

weakness N=50

right weakness N=56,

Bilateral N=1, no signs

N=1, presence of

weakness unknown N=2


M=10.5 (SD.5.8)

Recruited 4

weeks post-

stroke

Boosman et

al.3

BNI 54 Inclusion: Recently

suffered a stroke

(<12 months),

sufficient command

of the Dutch

language, Barthel

Score ≥ 19

Age: M= 53.8,

SD=12.3

Sex Male=64.8%,

Female=35.2%

Education:

High=33.3%,

Low=66.6%

Mechanism, Location and

Hemisphere:

Ischaemic=46.3%,

Left=44%, Right=44%,

Bilateral=8%.

Unknown=4%.

Haemorrhagic=11.1%,

Weeks: M=15.0,

SD=12.8

3

Exclusion: NR

Left=50%,

Right=33.3%,

Bilateral=16.7%.

Subarachnoid

haemorrhage=42.6%,

Anterior circulation

aneurysm=47.8%,

posterior circulation

aneurysm=30.4%,

Nonaneurysmal=21.7%

Severity: NR

Bour et al.4

MMSE 194 Inclusion: First ever

hemispheral stroke,

age above 40, native

speaker

Exclusion: Severe

aphasia, pre-stroke

dementia, and other

major neurological

or psychiatric

disorders that could

interfere with

testing.

Age: M=68.4,

SD=12.5

Sex: Male N=107,

Female N=89

Education:

high/low=86/107

Mechanism and location:

Territorial=39.2%,

Lacunar=52.6%,

Haemorrhagic=8.2 %

Hemisphere: Left=

41.6% , Right=56.8%,

Bilateral=1.6%

Severity: Rankin M=3.1,

SD=1.4

Within 1 month

Cartoni et

al.5

MEAMS 30 Inclusion: Stroke

patients within

rehabilitation ward

Exclusion: no other

neurological

impairments, no

mental

Age: M=75.80,

SD=7.98

Sex: 17 Male, 13

Female

Education: NR

Mechanism: NR

Location: TACI N=6,

PACI N=13, POCI N=8,

LACI N=3


Left N=14, Bilateral N=3.

Severity: NR

NR

4

confusion/dementia

prior to stroke. No

aphasia, visual

impairment or

auditory impairment

such that they could

not be tested

Cumming

et al.6

MMSE

MoCA

60 Inclusion: Ischaemic

or intracerebral

haemorrhagic stroke.

Exclusion: younger

than 18 years of age,

unconscious on

admission to

hospital, required an

interpreter, major

visual/hearing/

language impairment

Age: M=72.1,

SD=13.9

Sex: Male N=44,

Female N=16

Education: M=10.5,

SD=3.9

Mechanism: Infarct N=55,

Haemorrhage N=5

Location: TACI N=4,


LACI N=8


Left N=20, Bilateral N=2,

Unknown N=7

Severity: NIHSS median =

5

Days: M=98.3

SD=12.0

Godefroy et

al.7

MMSE

MoCA

MMSEadj

MoCAadj

(Adjusted

scores

created

using

control

95 Inclusion: Recent

infarct or

haemorrhage (< 3

weeks)

Exclusion: Severe

general and

neurological

conditions

precluding

neuropsychological

assessment,

illiteracy, mental

Age : M=68.2,

SD=13.7

Sex: Male N=60,

Female N=35

Education: primary

N=55, secondary

N=24, high school

N=16


Haemorrhage N=7

Location: ACA N=1

MCA N=65, PCA N=4,

Posterior Fossa N=18.

Haemorrhages: Lobar

N=3, Deep N=4.


Left N=39, Bilateral N=12

Severity: NIHSS M=2.7,

SD=3.2

Days: M=6.6,

SD=3.5

5

group to

account

for age

and

education)

retardation, non-

native speaker, other

neurological and

psychiatric

conditions

Grace et

al.8

MMSE

Modified

MMSE

70 Inclusion: Patients

with stroke

undergoing

treatment on a

medical

rehabilitation unit

Exclusion: NR

Age: M=74.5, SD=7.7

Sex: Male N=27,

Female N=43

Education: M= 10.4,

SD= 2.6.

Mechanism: NR

Location: NR

Hemisphere: Right N=41

Left N=15, bilateral N=8,

Non-focal lesion N=6.

Severity: NR

NR

Green et

al.9

RBANS 60 Exclusion: unable to

provide consent,

aphasia,

visual/auditory

impairments

precluding

completion of tests,

poor English,

diagnosis of

dementia, older than

80 years

Age: M=67.7,

SD=13.2

Sex: Male N=35,

Female N=25

Education: M= 10.6,

SD= 3.0

Mechanism: Ischemia:

N=45, Haemorrhage N=6,

Unspecified N=9.

Location: TACS N = 17,

PACS N = 15, POCS = 1,

LACS N =18


Left N=20, Bilateral: 2,

Unknown = 2

Severity: NR

Days: M=24,

SD=16.

Morris et

al.10

ACE-R

MMSE

61 Inclusion: NR.

Recruited from

stroke wards and

identified from

medical notes

Exclusion: History

of psychiatric

Age: Median=76.0,

IQR 67.0–82.5

Sex: Male N=31,

Female N=30

Education:

Median=9.0, IQR 9-

11

Mechanism: NR

Location: TACS N=15,

PACS N=10, POCS N=9,

LACS N=21,

Unknown N=6


Left N= 22, Unknown N=

3

Days: Median=

18, IQR 9-48-8

6

problems, blind/deaf,

too ill, too drowsy,

poor English,

moderate or severe

dysphasia


Median=9.0, IQR 6-12

Nøkleby et

al.11

Cognistat

SINS

Clock

Drawing

49 Inclusion: Diagnosis

of stroke, age > 18,

stable medical

condition, ability to

give informed

consent

Exclusion: NR

Age: Median=62,

IQR=53.5–77

Sex: Male N=32,

Female =17

Education: NR

Mechanism: Infarct N= 40,

Intracerebral Haemorrhage

N=6,

Subarachnoid

Haemorrhage N=3

Location: NR


Left N=19, Other N=6

Severity: NR

Days: Median

38, IQR 17-89

Nys et al.12

MMSE 34 Inclusion: Ischaemic

or haemorrhagic

stroke

Exclusion: High

degree of handicap,

non-native speaker,

severe disturbance in

communication and

consciousness, blind

Age: M=64.7,

SD=11.5

Sex: Male=41.2%

Education (scale):

Median=4


Haemorrhage N=1

Location: PACI – 5, LACI

– 21, POCI – 6, TACI – 1,

POCH - 1


Left N=17

Severity: Modified Rankin

Scale: Median=3

Days: M = 6.5,

SD = 2.9

Salvadori et

al.13

MoCA 80 Inclusion: Diagnosis

of stroke (ischemic

or haemorrhagic),

age 18+ years, native

speaker, informed

consent and

availability for

bedside evaluation

Age: M=68.2,

SD=14.6

Sex: Male=66%,

Female=33%

Education: No PCSI

M=10.9, SD=4.3,

PSCI M=7.9, SD=4.1

Mechanism: No PSCI

Ischemic=82%, PSCI

Ischaemic=89%

Location: No PSCI

Supratentorial=85%, PSCI

Supratentorial=85%

Hemisphere: No PSCI

Left=55%,

5-9 days post

admission

7

between 5th and 9th

day after stroke.

Exclusion: Patients

living outside the

Florence area

PSCI Left=51%

Severity: NIHSS: M=3.6,

SD=4.8

Schweizer

et al.14

MMSE

MoCA

32 Inclusion:

aneurysmal

subarachnoid

haemorrhage,

classified as good

outcome or moderate

disability according

to Glasgow Outcome

Scale

Age : M=55.2,

SD=7.8

Sex : Male N=13,

Female N=19

Education: M=15.8,

SD=3.8

Mechanism:

Aneurysm=100%

Location: Anterior

circulation N=18, MCA

N=4, Posterior circulation

N=9, Unknown N=1

Hemisphere: NR

Severity: WFNS grade: I

N=22, II N=3, III N= 3, IV

N=2

Months:

M=29.3,

SD=17.5

Srikanth et

al.15

S-MMSE

79 Inclusion: First ever

stroke, spoke

English, not severely

dysphasic, score ≥2

on the NIHSS,

adequate vision and

hearing.

Exclusion: NR

Age: M=69.0,

SD=14.4

Sex: Male N=47,

Female N=32

Education:

Cognitively intact

M=9.7 SD=2.2,

Cognitive impairment

no dementia M=10.0

SD=2.3, Dementia

M=9.9 SD=2.2

Mechanism: Ischaemic N=

72, Haemorrhagic N=7

Location: LACI N=26,


TACI N=2


Left N=25, Bilateral N=15

Severity: Barthel

Cognitively intact M=19.1

SD=1.7, Cognitive

impairment no dementia

M=19.6 SD=1.0,

Dementia M=17.6 SD=3.0

Days: M=381.6,

SD=45.6.

Wong et

al.16

MoCA

74 at 2-4 week

assessment

Inclusion:

spontaneous

subarachnoid

2-4 weeks

Age: Median=58

IQR=49-66

Mechanism:

Haemorrhage=100%

2-4 weeks

2-4 weeks & 1

year

8

ACA Anterior Cerebral Artery, ACE-R Addenbrooke’s Cognitive Examination – Revised, AUC Area Under Curve, BNI Barrow Neurological

Institute screen for higher cerebral functions, CDT Clock Drawing Task, ICA IntraCerebal Aneurysm, IQR Interquartile Range, LACI/S

Lacunar Infarct/Syndrome, M Mean, MEAMS Middlesex Elderly Assessment of Mental State, MCA Middle Cerebral Artery, MMSE Mini-

Mental State Examination, MMSE-S Standardised Mini-Mental State Examination, MoCA Montreal Cognitive Assessment, NIHSS National

Institute of Health Stroke Severity, N Sample Size, NR Not reported, PACI/S Partial Anterior Circulation Infarct/Syndrome, PCA Posterior

Cerebral Artery, PComA Posterior Communicating Artery, POCH Posterior Circulation Haemorrhage, POCI/S Posterior Circulation

MMSE 80 at 1-year

assessment

haemorrhage,

hospital admission

within 96 hours of

ictus, between 21

and 75 years of age,

native speaker,

willing and able to

provide informed

consent

Exclusion: history of

previous

cerebrovascular or

neurological disease

other than

unruptured

intracranial

aneurysm, a history

of neurosurgery

before ictus, inability

to cooperate with

cognitive

assessments (unable

to obey commands).

Sex: Female=50%

Education: NR

1-year

Age: Median=52

IQR=47-61

Sex: Female=55%

Education: NR

Location: ICA other than

PComA N= 11, PComA

N=13, ACA N=24, MCA

N=18, PCA N=8

Severity: WFNS Grade 1

N= 48, 2 N=15, 3 N=4, 4

N=6, 5 N=1

1-year

Location: ICA other than

PComA N=16, PComA

N=16, ACA N=26, MCA

N=20, PCA N=12

Severity: WFNS grade 1

N=45, 2 N=21, 3 N=1, 4

N=9, 5 N=4

Hemisphere: NR

9

Infarct/Syndrome, PSCI, Post Stroke Cognitive Impairment, RBANS Repeatable Battery for the Assessment of Neuropsychological Status, SD

Standard Deviation, SINS Neuropsychological Impairments in Stroke, TACI/S Total Anterior Circulation Infarct/Syndrome, WFNS World

Federation of Neurological Surgeons

10

Supplemental Table II. Evaluation of Research Methodologies

Study Sample

size

justificat

ion

Documented

recruitment

and attrition

Stroke

diagnosis

confirmed

with brain

imaging

Time between

screening and

gold-standard

assessment

Cognitive domains

included in gold

standard assessment

for SE and SP

analyses

Criteria for gold-

standard

impairment

Stratified

results for

demographic

and stroke

variables

Agrell et

al.1

NR Yes. No

differences in

sex, education

or diagnosis.

However,

study sample

had less left

sided lesions

and were

younger than

non-

participants

Diagnosis was

determined by

CT in 60% of

participants.

The remainder

were

determined by

clinical

investigation

NR Spatial memory

Arithmetic*

Verbal Memory

Spatial Function

Visual reasoning*

From

neuropsychological

tests an estimate

was made blindly

of whether a

patients was

mentally

unimpaired or

impaired

No lesion

hemisphere

effect for SE.

SP higher in

right

hemisphere

group.

Blake et

al.2

NR NR NR Within 3

months

Spatial perception

Visual inattention

Memory (verbal and

visual)

Apraxia

Executive functioning

Language

Clinician classified

patient as impaired

on at least one

cognitive domain

NR

Boosman

et al.3

NR NR NR Same day Perception

Language

Memory

Attention

Reasoning

Below 5th

percentile or within

the first decile on

one or more

cognitive domains.

Higher SE and

SP in people

aged over 55.

11

Executive Functioning Dutch norms used

when possible.

Bour et

al.4

Yes NR between

participants

and non-

participants.

Those who did

not complete

follow up were

older, less

educated and

had lower

original

MMSE scores

Yes Within 1

month

Memory

Mental Speed

Executive function

Calculation*

Visuospatial*

Orientation*

Attention*

Praxis*

Reasoning*

Language*

Performance below

10th

percentile

compared to

normative group.

Separate analyses

for 1,2, and 4

impaired domains

Results

comparable

across left and

right lesions

Cartoni et

al.5

NR Yes. However,

no statistical

comparison

between

original and

final sample.

NR Days:

M=20.73, SD=

24.37

Memory

Language

Perception

Apraxia


Overall clinician

conclusion from

report

NR

Cumming

et al.6

Yes Yes. However,

no statistical

comparison

between

original and

final sample.

NR Days: M=8.1,

SD 2.4

Visuospatial

Memory

Executive

Language

Attention

Visual neglect

Domain z-scores of

<−1 compared to

age/education

normative data in 2

or more domains

Follow up analyses

included more

stringent criterion

of z-score <-1.5.

MMSE: No

lesion

hemisphere

effect

MoCA: better

predictive

validity in

right lesion

participants

Godefroy

et al.7

NR NR NR Not directly

reported.

Language

Visuoconstructive

Impairment of at

least 2 cognitive

When

adjusting for

12

Estimated

M=17.5 days

abilities

Working memory

Long term memory

Action speed

Executive function

domains, at a

criterion that

corresponded to the

5% level (whether

compared to age

and education

normative data not

reported)

OR

MMSE score lower

than ≤ 23

age and

education,

MMSE SE

slightly

increased but

SP slightly

decreased.

MoCA SE and

SP showed

opposite trend.

Grace et

al.8

NR NR NR Within 1-

week, across

three sessions

Orientation and mental

control

Language

Verbal fluency

Visuospatial ability

Memory

Performance <2

SDs below

published norms in

two or more

cognitive domains.

Memory

performance was

defined as

impaired when it

fell at least 1.5 SDs

below published

norms

No lesion

hemisphere

effects for the

modified

MMSE.

Higher SE and

SP in left

hemisphere

lesions for

standard

MMSE.

Green et

al. 9

NR Yes. However,

no statistical

comparison

between

original and

final sample

NR Days: M=4,

SD = 5

Language

Perception

Attention

Memory

Executive abilities

Performance < 5th

percentile on any

subtest published

normative data

NR

Morris et NR Limited NR Days: Verbal memory Below 5th

NR

13

al.10

Median=3,

IQR 2−7.

Visual memory


Attention

Perception

percentile on one

or more cognitive

domains

Nøkleby

et al.11

Limited

rationale

provided

Limited WHO criteria

used

NR Language

Visuospatial

Function

Attention and neglect

Apraxia

Speed in unaffected

arm

Memory

At least one

cognitive domain

impaired (at least

1.5–2 SD from

estimated level

before stroke)

NR

Nys et

al.12

NR Yes. However,

no statistical

comparison

between

original and

final sample.

Yes Same day Abstract reasoning

Verbal memory

Executive function

Visual perception and

construction

Visual memory

Language

Performance below

-1.65 z-score in at

least one cognitive

domain compared

to matched controls

NR

Salvadori

et al.13

NR Yes. Patients

not included in

analyses had

lower pre-

morbid

cognitive

status, higher

stroke severity

and lower

MoCA scores

Yes Months:

M=8.4, SD 2.2

Verbal memory

Focalised and

maintained attention

Divided and selective

attention

Language

Abnormal

performance

(below 5th

percentile on age

and education

adjusted norms) on

one test and

borderline

performance on at

least one other test.

NR

Schweizer

et al.14

NR Yes. However,

no statistical

comparison

NR NR Attention

Executive function

Verbal learning and

2 SD below

age/education

normative data

NR

14

between

original and

final sample.

memory

Naming

Motor function

Each cognitive

domain

investigated

separately

Srikanth

et al.15

NR Yes. However,

no statistical

comparison

between

original and

final sample.

Yes NR General intellect

General

intellect/executive

ability

Verbal memory

Everyday memory

Spatial ability

Language/executive

ability

Scores more

than 1 SD below

age- and education-

corrected norms in

at least two tests

measuring the same

domain.

Participants with

dementia excluded

from ROC analysis

NR

Wong et

al.16

NR NR Yes During same

session or in 2

sessions on

consecutive

days

Verbal Memory

Visuospatial skills and

memory

Attention and working

memory


and psychomotor speed

Naming

A cognitive domain

deficit was defined

as a cognitive

domain z score ≤-

1.65.

Cognitive

impairment was

defined as two or

more cognitive

domain deficits

NR

* Limited assessment of these domains noted.

IQR Interquartile Range, NR Not reported, M Mean, MMSE Mini-Mental State Examination, MoCA Montreal Cognitive Examination, ROC

Receiver Operating Characteristics, SD Standard Deviation, WHO World Health Organisation

15

Supplemental Table III. Sensitivity and Specificity Data for Focal Cognitive Domains.

Screening Tool Gold Standard Assessment Domain

Memory Immediate Memory Delayed Memory Language

Subtest Cut-point SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV

MEAMS

Cartoni et

al. 5

Orientation Pass raw

score

57 80 NR NR

Remembering

Pictures

Pass raw

score

28 80 NR NR

Name Learning Pass raw

score

28 86 NR NR

Naming Pass raw

score

100 69 NR NR

Comprehension Pass raw

score

50 88 NR NR

Verbal fluency Pass raw

score

75 84 NR NR

Spatial

construction

Pass raw

score

Fragmented

letter

Pass raw

score

Unusual views Pass raw

score

Usual views Pass raw

score

Motor

perseveration

Pass raw

score

RBANS

Green et

al. 9

Immediate

Memory

79/80 84 71 56 91

Delayed

Memory

79/80 86 58 65 82

Language 79/80 88 91 77 95

Visuospatial 76/77

Attention 83/84

ACE-R

Morris et

Visuospatial 14

Fluency 8

16

al. 10

Memory 20 85 20 65 43

Attention and

Orientation

16

Cognistat

Nokleby

et al. 11

Memory >65yo 7/8

≤65yo

9/10

69 52 NR NR

Attention 5/6 54 73 NR NR

Similarities >65y 3/4

≤65y 4/5

60 82 NR NR

Comprehension 4/5 60 77 NR NR

Naming 6/7 80 77 NR NR

Repetition 10/11 60 82 NR NR

Visuo-

construction

>65yo 2/2

≤65yo 3/4

SINS

Nokleby

et al. 11

Aphasia 11/12 70 85 NR NR

Visuocognitive 17/18

Apraxia 20/21

17

Supplemental Table III. Sensitivity and Specificity Data for Focal Cognitive Domains (continued).

Screening Tool Gold Standard Assessment Domain

Perception/Visuospatial Executive Function Attention/Neglect Speed Praxis

Subtest Cut-

off

SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV

MEAMS

Cartoni

et al. 5

Orientation Pass

raw

score

Remember

Pictures

Pass

raw

score

Name

Learning

Pass

raw

score

Naming Pass

raw

score

Comprehen. Pass

raw

score

Verbal

fluency

Pass

raw

score

Spatial

construction

Pass

raw

score

44 100 NR NR

Fragmented

letter

Pass

raw

score

16 100 NR NR

Unusual

views

Pass

raw

score

61 91 NR NR

Usual views Pass

raw

score

55 75 NR NR

18

Motor

persever.

Pass

raw

score

11 100 NR NR

RBANS

Green et

al. 9

Immediate

Memory

79/80

Delayed

Memory

79/80

Language 79/80

Visuospatial 76/77 89 96 97 85

Attention 83/84 79 48 66 65

ACE-R

Morris et

al. 10

Visuospatial 14 88 47 76 67

Fluency 8 81 50 7 65

Memory 20

Attention

and

Orientation

16 92 44 29 96

Cognistat

Nokleby

et al. 11

Memory >65y

7/8

≤65y

9/10

Attention 5/6 80 65 NR NR

Similarities Age

depen

dent

Comprehen. 4/5

Naming 6/7 61 71 NR NR

Repetition 10/11

Visuo-

construction

>65y

2/3

≤65y

3/4

64 67 NR NR 64 58 NR NR 61 68 NR NR

SINS

Nokleby

et al. 11

Aphasia 11/12

Visuocog. 17/18 82 54 NR NR 72 42 NR NR

Apraxia 23/24 40 65 NR NR

19

All results presented as percentages. Due to large amount of data Confidence Intervals are not included in this table. Bolded data indicates results

reach traditionally acceptable levels of 80% SE and 60% SP.

MEAMS Middlesex Elderly Assessment of Mental State, NPV Negative Predictive Value, NR Not Reported, PPV Positive Predictive Value,

RBANS Repeatable Battery for the Assessment of Neuropsychological Status, SINS Neuropsychological Impairments in Stroke, SE Sensitivity,

SP Specificity

20

Supplemental Figure I. Search strategy example – OVID MEDLINE

21

Supplemental Figure II. Flowchart of Systematic Literature Review

22

Supplemental References

1. Agrell B, Dehlin O. Mini mental state examination in geriatric stroke patients. Validity, differences between subgroups of patients, and

relationships to somatic and mental variables. Aging Clin Exp Res. 2000;12:439-444

2. Blake H, McKinney M, Treece K, Lee E, Lincoln NB. An evaluation of screening measures for cognitive impairment after stroke. Age

Ageing. 2002;31:451-456

3. Boosman H, Visser-Meily JM, Post MW, Duits A, van Heugten CM. Validity of the barrow neurological institute (BNI) screen for

higher cerebral functions in stroke patients with good functional outcome. The Clin Neuropsychol. 2013;27:667-680

4. Bour A, Rasquin S, Boreas A, Limburg M, Verhey F. How predictive is the MMSE for cognitive performance after stroke? J Neurol.

2010;257:630-637

5. Cartoni A, Lincoln NB. The sensitivity and specificity of the middlesex elderly assessment of mental state (MEAMS) for detecting

cognitive impairment after stroke. Neuropsychol Rehabil. 2005;15:55-67

6. Cumming TB, Churilov L, Linden T, Bernhardt J. Montreal cognitive assessment and mini-mental state examination are both valid

cognitive tools in stroke. Acta Neurol Scand. 2013;128:122-129

7. Godefroy O, Fickl A, Roussel M, Auribault C, Bugnicourt JM, Lamy C, et al. Is the montreal cognitive assessment superior to the mini-

mental state examination to detect poststroke cognitive impairment? A study with neuropsychological evaluation. Stroke. 2011;42:1712-

1716

8. Grace J, Nadler JD, White DA, Guilmette TJ, Giuliano AJ, Monsch AU, et al. Folstein vs modified mini-mental state examination in

geriatric stroke: Stability, validity, and screening utility. Arch Neurol. 1995;52:477-484

9. Green S, Sinclair E, Rodger E, Birks E, Lincoln N. The Repeatable Battery for the assessment of Neuropsychological Status (RBANS)

for post-stroke cognitive impairment screening. Int J Ther Rehabil. 2013;20: 536-541.

10. Morris K, Hacker V, Lincoln NB. The validity of the addenbrooke's cognitive examination-revised (ACE-R) in acute stroke. Disabil

Rehabil. 2012;34:189-195

11. Nokleby K, Boland E, Bergersen H, Schanke AK, Farner L, Wagle J, et al. Screening for cognitive deficits after stroke: A comparison of

three screening tools. Clin Rehabil. 2008;22:1095-1104

12. Nys GM, van Zandvoort MJ, de Kort PL, Jansen BP, Kappelle LJ, de Haan EH. Restrictions of the mini-mental state examination in

acute stroke. Arch Clin Neuropsychol. 2005;20:623-629

13. Salvadori E, Pasi M, Poggesi A, Chiti G, Inzitari D, Pantoni L. Predicitver value of the MoCA an in the acute phase of stroke on the

diagnosis of mid-term cognitive impairment. J Neurol. 2013;260:2220-2227

14. Schweizer TA, Al-Khindi T, Macdonald RL. Mini-mental state examination versus montreal cognitive assessment: Rapid assessment

tools for cognitive and functional outcome after aneurysmal subarachnoid hemorrhage. J Neurol Sci. 2012;316:137-140

23

15. Srikanth V, Thrift AG, Fryer JL, Saling MM, Dewey HM, Sturm JW, et al. The validity of brief screening cognitive instruments in the

diagnosis of cognitive impairment and dementia after first-ever stroke. Int Psychogeriatr. 2006;18:295-305

16. Wong GK, Lam SW, Wong A, Ngai K, Poon WS, Mok V. Comparison of montreal cognitive assessment and mini-mental state

examination in evaluating cognitive domain deficit following aneurysmal subarachnoid haemorrhage. PLoS ONE. 2013;8:e59946

are cognitive screening tools sensitive and specific ... · keywords included stroke, cerebrovasc*,...

Documents