6 Major Flaws in 6 Major Flaws in Neuropsychological and Neuropsychological and

Psychodiagnostic ReportsPsychodiagnostic ReportsKyle Boone, Ph.D., ABPP-ABCNKyle Boone, Ph.D., ABPP-ABCN

California School of Forensic StudiesCalifornia School of Forensic StudiesAlliant International UniversityAlliant International University

May 11, 2015May 11, 2015

Report FlawsReport Flaws

1) Failure to 1) Failure to appropriately assess for appropriately assess for performance validityperformance validity

2) Failure to draw 2) Failure to draw conclusions consistent conclusions consistent with empirical researchwith empirical research

3) Failure to consider all 3) Failure to consider all possible etiologies for possible etiologies for cognitive abnormalitiescognitive abnormalities

4) Over-interpretation 4) Over-interpretation of lowered scoresof lowered scores

5) Claim that low 5) Claim that low cognitive scores cognitive scores document brain injurydocument brain injury

6) Misinterpretation of 6) Misinterpretation of personality test datapersonality test data

I. Failure to Appropriately Assess I. Failure to Appropriately Assess for Performance for Performance

Invalidity/Response BiasInvalidity/Response Bias A. Failure to detect response bias: A. Failure to detect response bias:

Administer zero, not enough, or Administer zero, not enough, or ineffective measuresineffective measures Current practice standards indicate that Current practice standards indicate that

formal measures of response bias are to be formal measures of response bias are to be interspersed interspersed throughoutthroughout neuropsychological neuropsychological examexam

NAN (Bush et al., 2005)NAN (Bush et al., 2005) Including use of Including use of embeddedembedded as well as as well as

free-standingfree-standing measures measures AACN (Heilbronner et al., 2009)AACN (Heilbronner et al., 2009)

Reliance on a single Performance Reliance on a single Performance Validity Measure (PVT) incorrectly Validity Measure (PVT) incorrectly

assumes that assumes that

Response bias is constant across an Response bias is constant across an examexam

Response bias presents in the same Response bias presents in the same manner in all individuals manner in all individuals i.e., that all patients use the same i.e., that all patients use the same

strategies when feigningstrategies when feigning

Instead:Instead:

Response bias typically fluctuates Response bias typically fluctuates across an examacross an exam

Even if response bias is constant, Even if response bias is constant, individuals differ in the strategies they individuals differ in the strategies they use when feigning cognitive symptomsuse when feigning cognitive symptoms

Therefore, need Therefore, need continuous samplingcontinuous sampling of performance validity using differing of performance validity using differing PVTsPVTs

Boone (2009)Boone (2009)

Response bias only during Response bias only during discretediscrete portions of exam:portions of exam:

Case #1: 51-year-old disability- seeking female claiming fibromyalgia, depression, and anxiety

Failed 2 PVTs half way through exam after she commented, “you do know that my brain is on overload!”

Case #2: 59-year-old disability-seeking male claiming panic attacks and depression

4 failed PVTs occurred only during 6 “panic attacks” in the exam

Response bias only during Response bias only during discretediscrete portions of exam: portions of exam:

Case #3: 45-year-old male litigant claiming chronic cognitive problems from mTBI

During morning session only failed 1 PVT, but after having lunch with his attorney, he failed all remaining effort indicators

Case #4: 31-year-old female litigant claiming chronic cognitive problems from mTBI

At start of exam claimed she was not “good” in the morning, and failed the first 2 PVTs;subsequent scores “zoomed” up to above average (FSIQ=145)

Response bias only during Response bias only during discretediscrete portions of exam: portions of exam:

Only at beginning of Only at beginning of exam exam

illustrating that she illustrating that she does not function in does not function in morningmorning

Only at end of examOnly at end of exam Illustrating that she Illustrating that she

cannot function when cannot function when tiredtired

Only after lunch Only after lunch meeting with attorneymeeting with attorney

Only during “panic Only during “panic attacks”attacks”

If PVTs had not If PVTs had not been been administered administered during these during these periods, response periods, response bias would not bias would not have been have been detecteddetected

Cognitive domains in which Cognitive domains in which symptoms can be faked:symptoms can be faked:

MemoryMemory AttentionAttention Mental SpeedMental Speed Language Language

(including reading)(including reading) MathMath

Visual Visual Perceptual/SpatialPerceptual/Spatial

IntelligenceIntelligence Motor Motor

dexterity/strengthdexterity/strength Any combination of Any combination of

the abovethe above

Response bias only on Response bias only on particularparticular taskstasks

Case #5

56-year-old female mTBI litigant

Failed PVTs reflecting motor/sensory function thinking speed visual perceptual/memory Standard cognitive scores normal with exception of above areas

Case #6

66-year-old male mTBI litigant

Failed PVTs reflecting verbal memory Standard cognitive scores normal with the exception of low average score in verbal memory

Response bias only on Response bias only on particularparticular tasks: tasks:

Case #7: Symptoms

PVTs Failed PVTs Passed

Primarily language symptoms that began days after the accident and progressively became more severe: Dysarthria /prominent (incon-sistent) articulation errors “Foreign accent” syndrome and ESLgrammatical errors: "How you say?" Word –retrievalproblems

Tests involving language and processing speed (2)

Noncredible on sensory exam:

Errors on tactile testing noncredible hearing results

Verbal memory (4)

Visual memory (1)

Attention/Math (2)

Motor speed (1)

Response bias only on Response bias only on particularparticular tasks: tasks:

In these 3 casesIn these 3 cases PVTs predicted which standard cognitive PVTs predicted which standard cognitive

scores were differentially loweredscores were differentially lowered If PVTs had not been administered If PVTs had not been administered

that covered these areas, that covered these areas, performance invalidity would not performance invalidity would not have been identifiedhave been identified

Noncredible patients are Noncredible patients are heterogeneousheterogeneous

There is no one “noncredible” profileThere is no one “noncredible” profile Some “malingerers” will do well on Some “malingerers” will do well on

some tests and this does not negate the some tests and this does not negate the fact that they are not crediblefact that they are not credible

I. Failure to Appropriately Assess I. Failure to Appropriately Assess for Performance Validityfor Performance Validity

B. Dismiss detected response bias:B. Dismiss detected response bias: Claim subject failed PVTs due to:Claim subject failed PVTs due to:

PainPain Depression/stress/anxiety/PTSDDepression/stress/anxiety/PTSD Pain or other medicationsPain or other medications FatigueFatigue Attentional lapsesAttentional lapses

Singly or in combinationSingly or in combination

Impact of Pain and Depression Impact of Pain and Depression on PVTson PVTs

But research shows thatBut research shows that acuteacute (Etherton et al., 2005a, b) and (Etherton et al., 2005a, b) and chronic chronic

(Iverson et al., 2007) (Iverson et al., 2007) painpain and and depressiondepression (see Goldberg, Back-Madruga, (see Goldberg, Back-Madruga,

& Boone, 2007, for review) & Boone, 2007, for review) do not lead to failure on PVTs do not lead to failure on PVTs All of the above symptoms are found inAll of the above symptoms are found in

credible credible patients with moderate to severe TBI patients with moderate to severe TBI on which the PVTs have been validated on which the PVTs have been validated

It would have to be argued that the It would have to be argued that the factors, singly or in combination, have factors, singly or in combination, have caused the person to have low cognitive caused the person to have low cognitive ability comparable to that found in people ability comparable to that found in people who do fail PVTs despite best effortwho do fail PVTs despite best effort The two primary groups who fail PVTs while The two primary groups who fail PVTs while

exerting best effort areexerting best effort are low IQlow IQ grossly impaired memorygrossly impaired memory (dementia, amnestic (dementia, amnestic

disorder)disorder)

If these conditions caused extremely If these conditions caused extremely low mental function, the effected low mental function, the effected people would lose the ability to drive, to people would lose the ability to drive, to care for themselves, etc., care for themselves, etc., ““To further place the patient’s performance in context, To further place the patient’s performance in context,

individuals with extremely low/mentally retarded IQ fail individuals with extremely low/mentally retarded IQ fail approximately 44% of effort indicators administered despite approximately 44% of effort indicators administered despite applying best effort (Dean et al., 2008), while the patient applying best effort (Dean et al., 2008), while the patient failed 91%. Thus, she performed worse than individuals with failed 91%. Thus, she performed worse than individuals with mental retardation yet she drives, parents, handles the mental retardation yet she drives, parents, handles the family finances, and grocery shops. The patient’s low family finances, and grocery shops. The patient’s low cognitive scores, if accurate, would in fact require that she cognitive scores, if accurate, would in fact require that she be reported to the DMV for removal of her license.” be reported to the DMV for removal of her license.”

Also, obviously, if such factors were Also, obviously, if such factors were to contaminate PVT performance, to contaminate PVT performance, they would also contaminate they would also contaminate standard cognitive test results, which standard cognitive test results, which therefore could not be used as therefore could not be used as indicative of the sequelae of any indicative of the sequelae of any frank brain injuryfrank brain injury

I. Failure to Appropriately Assess I. Failure to Appropriately Assess for Effort/Response Biasfor Effort/Response Bias

B. Dismiss detected response bias:B. Dismiss detected response bias: By pointing to PVTs that were passed, or By pointing to PVTs that were passed, or

intact performance on some standard intact performance on some standard cognitive taskscognitive tasks

However, cut-points are set to protect However, cut-points are set to protect credible patients (credible patients (<<10% false positives) at 10% false positives) at sacrifice to detection of noncredible patientssacrifice to detection of noncredible patients

Thus, failed scores are more informative than Thus, failed scores are more informative than passing scorespassing scores

As discussed earlier, the typical noncredible As discussed earlier, the typical noncredible patient is not underperforming on every taskpatient is not underperforming on every task

While it is not unusual for a credible While it is not unusual for a credible patient to fail a single PVT out of several patient to fail a single PVT out of several administered (with cut-offs set to administered (with cut-offs set to >>90% to 90% to <100% specificity)<100% specificity) only only 5% fail 25% fail 2 1.5% fail 31.5% fail 3 and and none fail 4 none fail 4

(Victor et al., 2009; see also Larrabee, 2003; Meyers (Victor et al., 2009; see also Larrabee, 2003; Meyers & Volbrecht, 2003; Sollman, Ranseen, & Berry, 2010)& Volbrecht, 2003; Sollman, Ranseen, & Berry, 2010)

Thus, what is important is not how many Thus, what is important is not how many are passed, but how many are failedare passed, but how many are failed

As analogy,As analogy, If there are If there are 10 banks 10 banks and a bank robber and a bank robber robs robs

only 4, only 4, would one conclude he/she is not a bank robber would one conclude he/she is not a bank robber

because 6 banks were not robbed?because 6 banks were not robbed?

I. Failure to Appropriately Assess I. Failure to Appropriately Assess for Effort/Response Biasfor Effort/Response Bias

B. Dismiss detected response bias:B. Dismiss detected response bias: By claiming that use of multiple PVTs By claiming that use of multiple PVTs

inflates false positive identificationsinflates false positive identifications Berthelson et al. (2013)Berthelson et al. (2013) Silk-Eglit et al. (2015)Silk-Eglit et al. (2015) Bilder et al. (2015)Bilder et al. (2015)

Silk-Eglit et al. (2015)Silk-Eglit et al. (2015) Using clinical sample concluded that to Using clinical sample concluded that to

maintain FP rate maintain FP rate <<10% when using 10% when using 33, , 77, , 1010, , 1414, and , and 1515 “embedded” PVTs, “embedded” PVTs,

Noncredible performance would be indicated by Noncredible performance would be indicated by failure on failure on >>11, , >>22, , >>33, , >>44, , >>55 PVTs, respectively PVTs, respectively

However, problems with study methodologyHowever, problems with study methodology mTBI litigants were allowed to fail 1 PVT, and PVTs used for group mTBI litigants were allowed to fail 1 PVT, and PVTs used for group

assignment had low sensitivity (Rey 15, TOMM), raising likelihood assignment had low sensitivity (Rey 15, TOMM), raising likelihood that noncredible subjects were included in the credible groupthat noncredible subjects were included in the credible group

Sample sizes small (24-25 per group)Sample sizes small (24-25 per group) Many of the embedded PVT scores were from the same test Many of the embedded PVT scores were from the same test

(therefore would be highly correlated and likely failed “as a group”)(therefore would be highly correlated and likely failed “as a group”)

Berthelson et al. (2013)Berthelson et al. (2013) Used a Monte Carlo simulation and concludedUsed a Monte Carlo simulation and concluded

If require 3 failures, not more than 8 PVTs can be If require 3 failures, not more than 8 PVTs can be administered without unacceptable FP rateadministered without unacceptable FP rate

RebuttalsRebuttals Davis and Millis (2014a)Davis and Millis (2014a) and and Larrabee (2014a)Larrabee (2014a)

In actual neurologic and clinical populations, rate of PVT In actual neurologic and clinical populations, rate of PVT failures was lower than predicted by Berthelson et al. (2013)failures was lower than predicted by Berthelson et al. (2013)

No significant relationship between number of PVTs No significant relationship between number of PVTs administered and number failed (r = .10) was foundadministered and number failed (r = .10) was found

Larrabee suggested that the simulation data were Larrabee suggested that the simulation data were problematic because test scores do not have the normal problematic because test scores do not have the normal distribution required for the analysis distribution required for the analysis

Rebuttal to RebuttalsRebuttal to Rebuttals Bilder, Sugar, and Hellemann (2014)Bilder, Sugar, and Hellemann (2014)

Asserted FP rate is elevated with use of Asserted FP rate is elevated with use of multiple PVTsmultiple PVTs

Suggested that practice of excluding low Suggested that practice of excluding low functioning samples from credible validation functioning samples from credible validation samples artificially lowers false positive ratessamples artificially lowers false positive rates

Recommended that before data on multiple Recommended that before data on multiple PVTs can be used clinically, empirical data are PVTs can be used clinically, empirical data are needed on various combinations of PVTs needed on various combinations of PVTs because of differing probabilities of joint failure because of differing probabilities of joint failure

Rebuttals to Rebuttal of RebuttalsRebuttals to Rebuttal of Rebuttals Davis and Millis (2014b) Davis and Millis (2014b)

Pointed out statistical limitations of the Bilder et Pointed out statistical limitations of the Bilder et al. analysesal. analyses

Argued that the standards Bilder et al. are Argued that the standards Bilder et al. are requiring for PVTs are not required of, or met by, requiring for PVTs are not required of, or met by, standard neuropsychological instrumentsstandard neuropsychological instruments

Showed that claimed large increase in FP rate with Showed that claimed large increase in FP rate with multiple PVTs is actually low in absolute numbersmultiple PVTs is actually low in absolute numbers

Predicted # PVT failures when 5 are administered is .55Predicted # PVT failures when 5 are administered is .55 Predicted # PVT failures when 9 are administered is 1.01Predicted # PVT failures when 9 are administered is 1.01

““doubling of error” but increase only from .5 to 1doubling of error” but increase only from .5 to 1

Rebuttals to Rebuttal of RebuttalsRebuttals to Rebuttal of Rebuttals Larrabee (2014b) Larrabee (2014b)

Argued that FP rates are elevated only in Argued that FP rates are elevated only in very low functioning patientsvery low functioning patients

Stroke with aphasiaStroke with aphasia TBI with imaging abnormalities and extensive TBI with imaging abnormalities and extensive

comacoma DementiaDementia Mental retardationMental retardation Severe psychiatric disturbanceSevere psychiatric disturbance

Additional IssuesAdditional Issues Test takers may elect to feign in particular cognitive Test takers may elect to feign in particular cognitive

domainsdomains Test taker failed 4 of 15 PVTs, but only in processing speed Test taker failed 4 of 15 PVTs, but only in processing speed

domain (4 of 6)domain (4 of 6) PVT failures may be extremePVT failures may be extreme

Test taker failed 4 of 12 PVTs – all in memory domain and some Test taker failed 4 of 12 PVTs – all in memory domain and some of the most extreme failures observedof the most extreme failures observed

Conclusion: test takers were feigning, but only Conclusion: test takers were feigning, but only in discrete domainsin discrete domains

Recommendations:Recommendations: Rather than simply summing PVTs, PVT failures should be Rather than simply summing PVTs, PVT failures should be

tabulated within cognitive domainstabulated within cognitive domains Extreme failures indicate noncredible performance regardless of Extreme failures indicate noncredible performance regardless of

number of PVTs administerednumber of PVTs administered

How to protect low functioning How to protect low functioning populationspopulations

Bilder et al. (2014) was critical of removing Bilder et al. (2014) was critical of removing low functioning individuals from credible low functioning individuals from credible samplessamples But the underlying assumption is incorrectBut the underlying assumption is incorrect

i.e., that a single cut-off could be developed for a i.e., that a single cut-off could be developed for a population ranging from very low functioning to high population ranging from very low functioning to high functioningfunctioning

Research shows that IQ is correlated with PVT Research shows that IQ is correlated with PVT performance in low IQ individuals, but not when performance in low IQ individuals, but not when IQ is low average or higher (e.g., Dean et al., IQ is low average or higher (e.g., Dean et al., 2008, Keary et al., 2013)2008, Keary et al., 2013)

How to protect low functioning How to protect low functioning populationspopulations

Best approach:Best approach: Remove low functioning subjects from primary PVT validation Remove low functioning subjects from primary PVT validation

studies and study them separatelystudies and study them separately Developing PVT cut-offs specific to the differential of actual Developing PVT cut-offs specific to the differential of actual

versus feigned low IQ versus feigned low IQ Smith et al., 2014Smith et al., 2014

55 credible low IQ (FSIQ 55 credible low IQ (FSIQ <<75) and74 noncredible 75) and74 noncredible with low IQ scores (FSIQ with low IQ scores (FSIQ <<75)75)

All PVT and neurocognitive cut-offs set to All PVT and neurocognitive cut-offs set to >>90% 90% specificity in credible samplespecificity in credible sample

When PVT failures were tabulated across 7 most When PVT failures were tabulated across 7 most sensitive PVTs (in this study)sensitive PVTs (in this study)

>>2 failures = 85% specificity, 87% sensitivity2 failures = 85% specificity, 87% sensitivity >>3 failures = 95% specificity, 66% sensitivity3 failures = 95% specificity, 66% sensitivity

11 Ethical Concerns regarding 11 Ethical Concerns regarding performance validity assessment performance validity assessment

Failing to use well-researched PVTsFailing to use well-researched PVTs Using PVTs only for defense casesUsing PVTs only for defense cases Using more or fewer PVTs, Using more or fewer PVTs,

systematically, depending on systematically, depending on whether you were retained by the whether you were retained by the defendant or plaintiffdefendant or plaintiff

Using different PVTs depending on Using different PVTs depending on which side retains youwhich side retains you

(Iverson, 2006)

Warning or prompting patients immediately before Warning or prompting patients immediately before administration of a PVTadministration of a PVT

Interpreting PVTs differently, systematically, Interpreting PVTs differently, systematically, depending on which side retains you (e.g., “cry for depending on which side retains you (e.g., “cry for help” if plaintiff-retained, malingering if defense-help” if plaintiff-retained, malingering if defense-retained)retained)

Assuming that someone who passes a PVT Assuming that someone who passes a PVT performed to true ability during the evaluationperformed to true ability during the evaluation

Interpreting PVT failure, in isolation, as malingeringInterpreting PVT failure, in isolation, as malingering Inappropriately interpreting PVT failure as a “cry for Inappropriately interpreting PVT failure as a “cry for

help”help” Competent, informed, and up-to-date use of tests Competent, informed, and up-to-date use of tests

(do not rely just on published test manuals)(do not rely just on published test manuals)

II. Failure to Draw Conclusions II. Failure to Draw Conclusions Consistent with Research Consistent with Research

Many reports conclude that observed Many reports conclude that observed cognitive abnormalities are due to cognitive abnormalities are due to long-term effects of mTBIlong-term effects of mTBI But a recent book summarizing the But a recent book summarizing the

research on mild traumatic brain injury research on mild traumatic brain injury (McCrea, 2007), (McCrea, 2007), published under the published under the auspices of the American Academy of auspices of the American Academy of Clinical Neuropsychology, concludesClinical Neuropsychology, concludes

““no indication of permanent impairment on no indication of permanent impairment on neuropsychological testing by three months neuropsychological testing by three months postinjury” postinjury” (p. 117)(p. 117)

Further, the following Further, the following meta-analytic meta-analytic studies studies show that there are show that there are no cognitive no cognitive abnormalities detected within days to abnormalities detected within days to monthsmonths after a mild TBI:after a mild TBI:

Belanger et al. (2005): 133 studies, n = 1463Belanger et al. (2005): 133 studies, n = 1463 Belanger and Vanderploeg (2005): 21 studies, n = Belanger and Vanderploeg (2005): 21 studies, n =

790790 Frencham et al. (2005): 17 studies, n = 634Frencham et al. (2005): 17 studies, n = 634 Schretlen and Shapiro (2003): 39 studies, n = 1716Schretlen and Shapiro (2003): 39 studies, n = 1716 Binder et al. (1997): 8 studies Rohling et al. (2011): 25 studies, n = 2828

Basis of the claimed Basis of the claimed 10%-15% 10%-15% mTBImTBI who do not recover? who do not recover?

Most influential publication:Most influential publication: Alexander (1995) published a review of mild Alexander (1995) published a review of mild

traumatic brain injury in which he stated traumatic brain injury in which he stated ““at 1 year after injury, 10 to 15% of mild TBI at 1 year after injury, 10 to 15% of mild TBI

patients have not recovered” patients have not recovered” and for which he provides two references: and for which he provides two references:

Rutherford, Merrett, and McDonald (1978) Rutherford, Merrett, and McDonald (1978) McLean et al. (1983) McLean et al. (1983)

However, examination of these publications However, examination of these publications shows that they do not support the above shows that they do not support the above statement statement

Rutherford et al. (1978)Rutherford et al. (1978)

Of 131 mild concussion patients, 14.5% still Of 131 mild concussion patients, 14.5% still reported symptoms at 1 yearreported symptoms at 1 year

However, However, “Of the 19 patients who had symptoms “Of the 19 patients who had symptoms at 1 year, at 1 year, 8 were involved in lawsuits8 were involved in lawsuits and and 6 had 6 had been suspected of malingeringbeen suspected of malingering 6 weeks after their 6 weeks after their accident. accident. Five of these patients were both Five of these patients were both involved in lawsuits and suspected of involved in lawsuits and suspected of malingering”malingering”

Further, info was recorded as Further, info was recorded as to “whether it was to “whether it was known that the patient was making a legal claim known that the patient was making a legal claim for compensation,” for compensation,” which suggests that in some which suggests that in some cases compensation-seeking was present but not cases compensation-seeking was present but not known to the examiners known to the examiners

Patients were asked to rate themselves on 16 Patients were asked to rate themselves on 16 symptoms, symptoms, including two cognitive categories: including two cognitive categories: loss of concentration and loss of memoryloss of concentration and loss of memory

only 3.1% (n = 4) reported loss of concentration and only 3.1% (n = 4) reported loss of concentration and 3.8% (n = 5) reported loss of memory. Thus, it 3.8% (n = 5) reported loss of memory. Thus, it would not be true that 10-15% reported continuing would not be true that 10-15% reported continuing cognitive symptoms; cognitive symptoms; <4% did <4% did

Further, the presence of symptoms was Further, the presence of symptoms was based based on patient self-report, not objective testingon patient self-report, not objective testing

McLean et al. (1983)McLean et al. (1983)

Very small sample (n = 20) of mostly mild TBI Very small sample (n = 20) of mostly mild TBI but with but with “a few cases” of mod/severe TBI“a few cases” of mod/severe TBI

compared to controls, the patients showed compared to controls, the patients showed “significant neuropsychological difficulties at “significant neuropsychological difficulties at 3 days, 3 days, but not at 1 month postinjurybut not at 1 month postinjury””

although the head injury sample endorsed more although the head injury sample endorsed more postconcussional symptoms at 1 monthpostconcussional symptoms at 1 month

Thus, a subset of mTBI patients may Thus, a subset of mTBI patients may reportreport more symptoms at one month, but this report more symptoms at one month, but this report is not corroborated by objective test results is not corroborated by objective test results

Dikmen and Levin (1993) note that studies Dikmen and Levin (1993) note that studies cited as documenting long term cognitive cited as documenting long term cognitive symptoms in mTBIsymptoms in mTBI ““were flawed by were flawed by inclusion of patients with inclusion of patients with

preexisting conditionspreexisting conditions (e.g., previous head (e.g., previous head injury) and failure to use appropriate controls injury) and failure to use appropriate controls to correct for these conditions” to correct for these conditions”

They suggest that “subsequent controlled They suggest that “subsequent controlled studies have studies have indicated indicated time-limited time-limited neuropsychological impairments that neuropsychological impairments that disappear by 1 to 3 months postinjurydisappear by 1 to 3 months postinjury” ”

What about impact of What about impact of multiple concussions?multiple concussions?

Some argue that while a single Some argue that while a single concussion may not result in permanent concussion may not result in permanent cognitive sequelae, more than one does, cognitive sequelae, more than one does, i.e., that while the mTBI associated with the i.e., that while the mTBI associated with the

accident in question may not have resulted accident in question may not have resulted in cognitive problems in a person with no in cognitive problems in a person with no history of TBI, history of TBI,

the fact that the plaintiff had a previous concussion the fact that the plaintiff had a previous concussion rendered him/her an rendered him/her an “eggshell” “eggshell” plaintiff who was plaintiff who was predisposed to chronic cognitive problems from any predisposed to chronic cognitive problems from any subsequent mTBIsubsequent mTBI

What does the literature say?What does the literature say?

Most investigations have found Most investigations have found no relationship no relationship between number of between number of concussions and cognitive test concussions and cognitive test performance performance

Collie, McCrory, and Makdissi (2006)Collie, McCrory, and Makdissi (2006) Guskiewicz, Marshall, Broglio, Cantu, and Guskiewicz, Marshall, Broglio, Cantu, and

Kirkendall (2002)Kirkendall (2002) Iverson, Brooks, Lovell, and Collins (2006);Iverson, Brooks, Lovell, and Collins (2006); Pellman, Lovell, Viano, Casson, and Tucker Pellman, Lovell, Viano, Casson, and Tucker

(2004) (2004)

What does the literature say?What does the literature say? Bijur, Haslum, and Golding (1996) Bijur, Haslum, and Golding (1996)

found that increasing numbers of mTBI in found that increasing numbers of mTBI in children were significantly related to lowered children were significantly related to lowered scores on measures of intelligence, and scores on measures of intelligence, and reading and math, reading and math,

but the same negative impact on cognition was but the same negative impact on cognition was found for number of non brain-injury traumasfound for number of non brain-injury traumas

leading the authors to conclude that leading the authors to conclude that ““cognitive deficits associated with multiple mild cognitive deficits associated with multiple mild

head injury are due to social and personal factors head injury are due to social and personal factors related to multiple injuries and not to specific related to multiple injuries and not to specific damage to the head” damage to the head”

What does the literature say?What does the literature say? Recent meta-analysis comparing effects of Recent meta-analysis comparing effects of

one self-reported TBI versus more than one one self-reported TBI versus more than one (Belanger et al.,2010), found that the(Belanger et al.,2010), found that the ““overall effect of multiple mTBI on overall effect of multiple mTBI on

neuropsychological functioning was minimal (d neuropsychological functioning was minimal (d = .06) and not significant= .06) and not significant”;”;

in examining specific cognitive domains, poorer in examining specific cognitive domains, poorer performance with multiple TBI was found on performance with multiple TBI was found on measures of delayed memory and executive measures of delayed memory and executive functioning, although effect sizes were small (functioning, although effect sizes were small (d = d = .16 .16 and .24, respectively) and and .24, respectively) and “their clinical “their clinical significance is unclear” significance is unclear”

Conclusions re: mTBIConclusions re: mTBI

No credible evidence of long-term No credible evidence of long-term cognitive compromise, even in those cognitive compromise, even in those with histories of more than one with histories of more than one concussionconcussion

III. Failure to Consider All III. Failure to Consider All Possible EtiologiesPossible Etiologies

Premature foreclosurePremature foreclosure:: ““a common mistake in clinical practice is a common mistake in clinical practice is

automatically to attribute the cause of the automatically to attribute the cause of the difficulties observed in patients seen long after the difficulties observed in patients seen long after the injury to the head injury” injury to the head injury”

““learning disabilities, psychiatric problems, learning disabilities, psychiatric problems, neurological disorders (e.g., epilepsy), and neurological disorders (e.g., epilepsy), and particularly previous head injuries and alcohol particularly previous head injuries and alcohol abuse are prevalent in the population with head abuse are prevalent in the population with head injury … these conditions in themselves are known injury … these conditions in themselves are known to be associated with neuropsychological and to be associated with neuropsychological and psychosocial problems”psychosocial problems”

(Dikmen & Levin, 1993)(Dikmen & Levin, 1993)

Conditions/characteristics that can Conditions/characteristics that can be associated with lowered be associated with lowered

cognitive scorescognitive scores Substance abuse by patient or exposure in Substance abuse by patient or exposure in

uteroutero Chronic medical illnesses such as hypertension, Chronic medical illnesses such as hypertension,

diabetes, sleep apnea, COPD, HIV, hepatitisdiabetes, sleep apnea, COPD, HIV, hepatitis Learning disability or attention deficit disorderLearning disability or attention deficit disorder Low educational level or history of special Low educational level or history of special

educationeducation MedicationsMedications Psychiatric conditions – depression, psychosisPsychiatric conditions – depression, psychosis Neurologic conditions – brain infections, Neurologic conditions – brain infections,

moderate to severe TBI, progressive dementiamoderate to severe TBI, progressive dementia Language (e.g., ESL) and cultural issuesLanguage (e.g., ESL) and cultural issues

All of the above have a more major impact All of the above have a more major impact on cognitive scores than mTBIon cognitive scores than mTBI

Effect Sizes on Cognition Effect Sizes on Cognition (Iverson, 2006)(Iverson, 2006)

Does mTBI predispose to Does mTBI predispose to depression?depression?

Recent meta-analysis of the relationship Recent meta-analysis of the relationship between mTBI and psychiatric symptoms between mTBI and psychiatric symptoms ((depressiondepression, , anxietyanxiety, , psychosocial disabilitypsychosocial disability, , reduced copingreduced coping)) 11 studies were suitable for inclusion and represented a total of 352 11 studies were suitable for inclusion and represented a total of 352

mTBI patients and 765 controlsmTBI patients and 765 controls Effect sizes were smaller when studies were weighted, indicating that Effect sizes were smaller when studies were weighted, indicating that

unweighted effect sizes were unduly influenced by studies with small unweighted effect sizes were unduly influenced by studies with small n’s and highly variable findingsn’s and highly variable findings

Effect sizes ranged from -.28 to .26, did not significantly differ from Effect sizes ranged from -.28 to .26, did not significantly differ from zero (p = .76), and were considered zero (p = .76), and were considered “meaningless”“meaningless”

The authors concluded that The authors concluded that “mTBI may have a very small to no “mTBI may have a very small to no measurable effect on psychological and psychosocial symptom measurable effect on psychological and psychosocial symptom reporting” reporting” Panayiotou, Jackson, and Crowe (2010)Panayiotou, Jackson, and Crowe (2010)

In ConclusionIn Conclusion

It is imperative to obtain a complete It is imperative to obtain a complete history regardinghistory regarding medical conditionsmedical conditions psychiatric conditionspsychiatric conditions education/occupationeducation/occupation

and integrate this information into and integrate this information into report conclusionsreport conclusions

IV. Over-interpretation of IV. Over-interpretation of Lowered ScoresLowered Scores

A. Failure to consider normal A. Failure to consider normal variability variability ¾ of normal volunteers obtained 1 ¾ of normal volunteers obtained 1

borderline to impaired score in test borderline to impaired score in test battery, and 20% obtained battery, and 20% obtained >>2 2 impaired scores impaired scores

Palmer et al. (1998)Palmer et al. (1998)

IV. Over-interpretation of IV. Over-interpretation of Lowered ScoresLowered Scores

A. Failure to consider normal variability A. Failure to consider normal variability Marked intraindividual variability is Marked intraindividual variability is

common in normal adultscommon in normal adults z-score discrepancies ranged from 1.6 SD to z-score discrepancies ranged from 1.6 SD to

6.0 SD; 66% of subjects had discrepancy 6.0 SD; 66% of subjects had discrepancy values that exceeded 3 SDsvalues that exceeded 3 SDs

Schretlen, Munro, Anthony, and Pearlson (2003)Schretlen, Munro, Anthony, and Pearlson (2003)

Review article: “abnormal performance on Review article: “abnormal performance on some proportion of neuropsychological some proportion of neuropsychological tests is psychometrically normal”tests is psychometrically normal”

Binder, Iverson, and Brooks (2009)Binder, Iverson, and Brooks (2009)

IV. Over-interpretation of IV. Over-interpretation of Lowered ScoresLowered Scores

B.) Incorrectly assume that all B.) Incorrectly assume that all claimants were at least average claimants were at least average before the injurybefore the injury

25% of the population are low 25% of the population are low average IQ or loweraverage IQ or lower

These individuals are not protected These individuals are not protected from injuryfrom injury

Premorbid function can be estimated Premorbid function can be estimated from preinjury educational and from preinjury educational and occupational backgroundoccupational background

IV. Over-interpretation of IV. Over-interpretation of Lowered ScoresLowered Scores

C.) Refer to low average scores (9C.) Refer to low average scores (9thth--2424thth percentile) as “impairments” percentile) as “impairments”

16%16% of normal population obtains of normal population obtains scores at this levelscores at this level

Better to use IQ labels so a common rubric Better to use IQ labels so a common rubric is employed across tests:is employed across tests:

Impaired = Impaired = <<22ndnd percentile percentile Borderline impaired = 3Borderline impaired = 3rdrd-8-8thth percentile percentile Low Average = 9Low Average = 9thth-24-24thth percentile percentile Average = 25Average = 25thth-74-74thth percentile percentile High Average = 75High Average = 75thth-90-90thth percentile percentile Superior = 91Superior = 91stst – 97 – 97thth percentile percentile Very Superior = Very Superior = >>9898thth percentile percentile

IV. Over-interpretation of IV. Over-interpretation of Lowered ScoresLowered Scores

D.) incorrectly assume that D.) incorrectly assume that individuals of above average individuals of above average intelligence should score above intelligence should score above average on other neurocognitive average on other neurocognitive teststests In the Palmer et al. (1998) study In the Palmer et al. (1998) study cited above, subjects had a mean cited above, subjects had a mean IQ in the high average rangeIQ in the high average range

¾ of normal volunteers obtained 1 ¾ of normal volunteers obtained 1 borderline to impaired score in test borderline to impaired score in test battery, and 20% obtain at least 2 battery, and 20% obtain at least 2 impaired scores impaired scores

IQ scores are not good predictors of cognitive IQ scores are not good predictors of cognitive function when individuals are above average in function when individuals are above average in intelligenceintelligence multiple studies have shown that individuals with high multiple studies have shown that individuals with high

intelligence do not obtain uniformly elevated scores on intelligence do not obtain uniformly elevated scores on cognitive exam:cognitive exam:

Diaz-Asper, Schretlen, and Pearlson (2004)Diaz-Asper, Schretlen, and Pearlson (2004) Hawkins and Tulsky (2001)Hawkins and Tulsky (2001) Russell (2001)Russell (2001)

leading Greiffenstein (2008) to conclude that the belief leading Greiffenstein (2008) to conclude that the belief that above average scores should be consistently that above average scores should be consistently found across cognitive tasks in individuals with above found across cognitive tasks in individuals with above average IQ is a neuropsychological average IQ is a neuropsychological “myth.”“myth.”

In a particularly relevant study, 20 professors with Ph.D. degrees and with negative medical and psychiatric histories were administered neuropsychological exams as a part of a research project

65% obtained at least 1 average score 30% had at least 1 low average score 10% had at least 1 borderline score 15% obtained an impaired score

Zakzanis & Jeffay (2011)

V. Claim that Low Cognitive V. Claim that Low Cognitive Scores Document Brain InjuryScores Document Brain Injury

Some clinicians reason that if a mild Some clinicians reason that if a mild traumatic brain injury patient is still traumatic brain injury patient is still showing cognitive abnormalities on a showing cognitive abnormalities on a long-term basis, this must prove that long-term basis, this must prove that the initial injury was more severe than the initial injury was more severe than a mild injurya mild injury ““The patient shows low memory and The patient shows low memory and

executive scores on testing (3 years post executive scores on testing (3 years post accident), which suggests that the original accident), which suggests that the original brain injury was more than mild”brain injury was more than mild”

But as Dikmen and Levin (1993) note, this But as Dikmen and Levin (1993) note, this line of reasoning line of reasoning

““tends to confuse severity with outcome or tends to confuse severity with outcome or independent variables with dependent variables” independent variables with dependent variables”

Determination of severity of traumatic brain Determination of severity of traumatic brain injury is based on injury characteristics at injury is based on injury characteristics at the time of the injury, the time of the injury, notnot cognitive testing cognitive testing results remote from the injuryresults remote from the injury

Ever seen a TBI study in which severity was Ever seen a TBI study in which severity was determined by cognitive scores remote from determined by cognitive scores remote from injury? injury?

TBI ClassificationTBI Classification

Mild Moderate Severe

GCS >13 9-12 <9

LOC<30 min.

>30 min to <24 hours

>24 hours

PTA <1 day>1 and <7

days>7 days

GCS = Glasgow Coma ScaleLOC = Loss of ConsciousnessPTA = Post traumatic amnesia

VI. Misinterpretation of the VI. Misinterpretation of the MMPI-2/RFMMPI-2/RF

Myths or Facts?Myths or Facts? 1) In personal injury litigants, elevations on 1) In personal injury litigants, elevations on

somatic complaints scales are consistent somatic complaints scales are consistent with expected concern over the injuries with expected concern over the injuries sustained in the accidentsustained in the accident ““Objective testing data revealed an individual Objective testing data revealed an individual

who is experiencing ….somatic or bodily who is experiencing ….somatic or bodily preoccupation, not unlike many individuals with preoccupation, not unlike many individuals with history of traumatic illnesses or injuries, history of traumatic illnesses or injuries, consistent with sequelae of traumatic brain consistent with sequelae of traumatic brain injury.”injury.”

VI. Misinterpretation of the VI. Misinterpretation of the MMPI-2/RFMMPI-2/RF

Myths or Facts?Myths or Facts? 2) The hypochondriasis/somatic 2) The hypochondriasis/somatic

complaints scales were not developed complaints scales were not developed on medical/neurologic patients and on medical/neurologic patients and should not be used in this populationshould not be used in this population

3) Elevations on validity scales indicate 3) Elevations on validity scales indicate a “cry for help” rather than malingeringa “cry for help” rather than malingering

4) The FBS scale misdiagnoses persons 4) The FBS scale misdiagnoses persons with actual disabilities as malingeringwith actual disabilities as malingering

Myths #1 and #2:Myths #1 and #2:Elevations on Somatic Complaints scales do not reflect overreport in Elevations on Somatic Complaints scales do not reflect overreport in injured litigants, and the scales were not developed/validated on true injured litigants, and the scales were not developed/validated on true

medical patients and therefore should not be used in medical medical patients and therefore should not be used in medical populationspopulations

MMPI data for a sample of 74 mixed chronic MMPI data for a sample of 74 mixed chronic neurologic patients (with diagnoses confirmed neurologic patients (with diagnoses confirmed by neurologic exam and objective tests, e.g., by neurologic exam and objective tests, e.g., MRI, EEG), MRI, EEG), mean Hs T score was 65 mean Hs T score was 65 (SD = 15) (cut-off >70)(SD = 15) (cut-off >70) mean Hy T score was 66 mean Hy T score was 66 (SD = 13) (cut-off >70) (SD = 13) (cut-off >70)

confirming that markedly elevated confirming that markedly elevated scores are not typical in this population scores are not typical in this population

Cripe, Maxwell, and Hill (1995)Cripe, Maxwell, and Hill (1995)

Available evidence suggests that 1-3 Available evidence suggests that 1-3 code-type likely code-type likely predatespredates the injury in the injury in persistent post-concussion syndromepersistent post-concussion syndrome

Greiffenstein and Baker (2001)Greiffenstein and Baker (2001)

Myths #1 and #2:Myths #1 and #2:Elevations on Somatic Complaints scales do not reflect overreport in Elevations on Somatic Complaints scales do not reflect overreport in injured litigants, and the scales were not developed/validated on true injured litigants, and the scales were not developed/validated on true

medical patients and therefore should not be used in medical medical patients and therefore should not be used in medical populations populations

MMPI-2-RF data for mixed neurologic (n = MMPI-2-RF data for mixed neurologic (n = 28), epilepsy (n = 50), and TBI (passing 28), epilepsy (n = 50), and TBI (passing PVTs; n = 27) patients revealed PVTs; n = 27) patients revealed all mean validity scores below cut-offs (i.e., all mean validity scores below cut-offs (i.e.,

<70T) <70T)

confirming that markedly elevated confirming that markedly elevated scores are not typical in these scores are not typical in these populations populations

Schroeder et al. (2012)Schroeder et al. (2012)

Development of Hypochondriasis Scale Development of Hypochondriasis Scale (CS1)(CS1)

Hypochondriasis scale was developed Hypochondriasis scale was developed on 4 groups (see Greene, 1991):on 4 groups (see Greene, 1991): NormalsNormals Individuals diagnosed as hypochondriacs Individuals diagnosed as hypochondriacs

by treating therapists by treating therapists Psychiatric patientsPsychiatric patients Medical patientsMedical patients

The final scale differentiated hypo-The final scale differentiated hypo-chondriacal group from chondriacal group from allall others others

The hypochondriasis scale was the first The hypochondriasis scale was the first clinical scale developed, indicating that clinical scale developed, indicating that differentiation of actual medical patients differentiation of actual medical patients from hypochondriacal patients was of from hypochondriacal patients was of high priority to the MMPI creatorshigh priority to the MMPI creators

A “hypochondriasis” scale that failed to A “hypochondriasis” scale that failed to distinguish actual medical patients from distinguish actual medical patients from hypochondriacs would be of little usehypochondriacs would be of little use

Myth #3: Myth #3: Elevated Validity scales = “cry for help”Elevated Validity scales = “cry for help”

Some argue that elevated validity scales Some argue that elevated validity scales represent an attempt by patients to insure represent an attempt by patients to insure that their psychological distress is noted that their psychological distress is noted ““Cry for help” Cry for help” was coined to describe those was coined to describe those

patients who appeared to be patients who appeared to be feigning/exaggerating psychiatric symptoms feigning/exaggerating psychiatric symptoms on the MMPI in on the MMPI in the absence of any the absence of any apparent external goal apparent external goal (Berry et al., 1996)(Berry et al., 1996)

Therefore, would not be appropriate for Therefore, would not be appropriate for use in settings where there is external use in settings where there is external incentiveincentive

What is empirical underpinning What is empirical underpinning for “cry for help” conclusion?for “cry for help” conclusion?

Search of pubmed located only Search of pubmed located only 3 studies:3 studies: Rogers et al. (1995): Rogers et al. (1995):

Psychiatric outpatients were asked to complete the MMPI-Psychiatric outpatients were asked to complete the MMPI-2 in an honest condition and then when simulating the 2 in an honest condition and then when simulating the goal of immediate hospitalization for severe psychiatric goal of immediate hospitalization for severe psychiatric problems. In the second condition, significantly higher problems. In the second condition, significantly higher scores were found on all F-family over-reporting scalesscores were found on all F-family over-reporting scales

Berry et al. (1996): Berry et al. (1996): Psychiatric clinic patients given a scenario in which they Psychiatric clinic patients given a scenario in which they

were experiencing significant psychiatric symptoms and were experiencing significant psychiatric symptoms and placed on a waiting list; they were told to complete the placed on a waiting list; they were told to complete the MMPI-2 in a manner that would enable them to receive MMPI-2 in a manner that would enable them to receive treatment more quickly. Their MMPI-2 pattern was treatment more quickly. Their MMPI-2 pattern was indistinguishable from that seen in frank malingerersindistinguishable from that seen in frank malingerers

Why did these studies observe a Why did these studies observe a “malingering” “malingering” profile?profile?

Because the subjects were asked to Because the subjects were asked to malinger,malinger, i.e., to deliberately feign i.e., to deliberately feign symptoms in the service of an external symptoms in the service of an external goalgoal

Third study:Third study:

Post and Gasparikova-Krasnec (1979)Post and Gasparikova-Krasnec (1979) 20 psychiatric inpatients who obtained MMPI F-K 20 psychiatric inpatients who obtained MMPI F-K

scores >11 (referred to as a “plea for help”) scores >11 (referred to as a “plea for help”) showedshowed

poorer impulse control and more “acting out” on the poorer impulse control and more “acting out” on the unit (sexual acting out, aggression, self-inflicted unit (sexual acting out, aggression, self-inflicted physical harm)physical harm)

more requirements for seclusionmore requirements for seclusion caused more “feelings of frustration” in staff caused more “feelings of frustration” in staff

Thus, it appears that the over-reporters had the Thus, it appears that the over-reporters had the tell-tale signs of tell-tale signs of borderline personality disorderborderline personality disorder

So, if a report were to refer to a “cry for help”, it would So, if a report were to refer to a “cry for help”, it would also need to indicate the likely presence of BPDalso need to indicate the likely presence of BPD

Greene (1988) initially raised concerns Greene (1988) initially raised concerns regarding the concept of “cry for help” regarding the concept of “cry for help” he noted that patients identified as he noted that patients identified as

overreporters on the MMPI were actually overreporters on the MMPI were actually lessless likely to follow through with treatment likely to follow through with treatment than individuals not showing the “cry for than individuals not showing the “cry for help” pattern, and in fact typically only help” pattern, and in fact typically only attended a single therapy sessionattended a single therapy session

That is, it can be questioned whether they were That is, it can be questioned whether they were engaging in a “cry for help” when in fact they engaging in a “cry for help” when in fact they refused the proffered helprefused the proffered help

Conclusions regarding Conclusions regarding “Cry for Help”“Cry for Help”

No empirical evidence for a nonconscious No empirical evidence for a nonconscious “cry for help” F-family scale pattern of “cry for help” F-family scale pattern of elevations on the MMPI-2 used to flag elevations on the MMPI-2 used to flag extent of psychological distress extent of psychological distress

Available evidence indicates that marked Available evidence indicates that marked elevations on F-family scales are elevations on F-family scales are associated with deliberate, motivated associated with deliberate, motivated feigning of symptoms, and in those cases feigning of symptoms, and in those cases when it may not be, it appears to be when it may not be, it appears to be related to borderline personality disorderrelated to borderline personality disorder

Myth #4: Myth #4: FBS misidentifies credible patients as malingerersFBS misidentifies credible patients as malingerers

FBS does not have a high false FBS does not have a high false positive ratepositive rate Using recommended cut-off of Using recommended cut-off of >>28 (raw)28 (raw), ,

false positive rate is <2% across patients false positive rate is <2% across patients with severe TBI, psychiatric disorders, with severe TBI, psychiatric disorders, medical/neurologic illness, substance medical/neurologic illness, substance abuse, brain disease, and epilepsyabuse, brain disease, and epilepsy

Scores above 30 Scores above 30 (raw on MMPI-2) never (raw on MMPI-2) never or rarely produce false positive errorsor rarely produce false positive errors

Greiffenstein, Fox, and Lees-Haley (2007)Greiffenstein, Fox, and Lees-Haley (2007)

FBS does not have a high false FBS does not have a high false positive ratepositive rate Studies that report high false positive Studies that report high false positive

rates rates have not excluded subjects with have not excluded subjects with motive to feignmotive to feign

See Larrabee (2003) for critiqueSee Larrabee (2003) for critique

Conclusions: What to Look For In Conclusions: What to Look For In a Neuropsychological Reporta Neuropsychological Report

Were data obtained on several measures of Were data obtained on several measures of response bias/performance validity?response bias/performance validity?

Is observed cognitive profile consistent with Is observed cognitive profile consistent with published literature for the condition?published literature for the condition?

Have all plausible causes for the cognitive Have all plausible causes for the cognitive abnormalities been considered?abnormalities been considered?

Have cognitive scores been interpreted in light of Have cognitive scores been interpreted in light of evidence as to how the patient functioned evidence as to how the patient functioned premorbidly and has normal variability in test premorbidly and has normal variability in test scores been considered?scores been considered?

Have raw scores been correctly interpreted (in Have raw scores been correctly interpreted (in terms of impaired, low average, etc., labels)?terms of impaired, low average, etc., labels)?

Have personality test results been correctly Have personality test results been correctly interpreted?interpreted?

Take home message:Take home message:

Conclusions contained in Conclusions contained in neuropsychological reportsneuropsychological reports

Need to be “Need to be “evidence-basedevidence-based””

i.e., grounded in the empirical i.e., grounded in the empirical literatureliterature

Questions?Questions?