t.o.s.s.a test of sustained selective...

T.O.S.S.A

Test of Sustained

Selective Attention

For Windows® 9X/ME/2000/NT/XP/Vista/7/10

version 4.0

MANUAL

Copyright © 2019

F. Kovács

TOSSA Manual: Test of Sustained Selective Attention version 4.0.4 May 2019

_______________________________________________________________________________________2

_________________________________________________________________________________________

© 2019 Pyramid Productions

Contents

1. Short manual: quick start……..………………………………………………………………. 3

1.1. Description of the test and the TOSSA indices……….………………….. 3

System requirements………………………………………………….………. 3

1.2. Administering the TOSSA: procedures and instructions....................... 5 1.2.1. Starting the TOSSA……………………………………………………… 5 1.2.2. Instructions……………………………………………………………….. 6 1.2.3. Storing the TOSSA results: manual setup……...…………………........... 8

1.2.4. Tips for an optimal administration of the TOSSA……………………. 11 1.2.5. Interpreting the TOSSA results……………………………………… 12

2. Theoretical background….……………………………………………………………………. 15 3. Norm research and psychometric characteristics……...………………………………… 19 3.1. Research in norming the TOSSA..……………………………………………….. 19 3.2. Intermezzo about statistics: normal distributions and probability calculations………..…………………………………………………………………. 35 3.3. Discriminative power of the TOSSA: sensitivity and specificity………….... 36

3.4. Reliability and validity………..…………………………………………................ 44

3.4.1. Reliability………………………………………………………………….. 44 3.4.2. Validity….………………………………………………………................ 45 3.4.2.1. Convergent validity of the TOSSA……….……………….. 50 3.4.2.2. Divergent validity of the TOSSA……….………………….. 51

4. Possible criteria to recognize reduced effort or malingering….…………….…………. 55

5. TOSSA versus other attention tests………..……………………………………………….. 57 6. Literature…………………………………………………………………………………………. 59

Appendix I: coding system for education and diagnosis….…..………………….. 61


_______________________________________________________________________________________3

_________________________________________________________________________________________


1. Short manual: Quick start! ______________________________________________________

1.1. Description of the test and the TOSSA indices The Test of Sustained Selective Attention (TOSSA) consists of 5 files: - TOSSA.EXE : the executable program - PIEP2.WAV : a WAV-file of 2 beeps - PIEP3.WAV: a WAV-file of 3 beeps - PIEP4.WAV: a WAV-file of 4 beeps - NORMGRAPHTOSSA.TXT : data of the norm graph of the TOSSA (do NOT change it!!) In the install directory more files are visible but these are part of the security program connected to the TOSSA: !! TOSSA.EXE.CM !! CMINSTALL.EXE (or CMSERVER if it is a Network installation) !! TOSSA.EXE.CM.INI

System requirements: The TOSSA has been developed for MS-WINDOWS 95/98/ME/2000/NT/XP/Vista/7/10 and runs at least on a Pentium II 300 MHz computer with a Soundblaster compatible sound card. A mouse is advisable but not strictly necessary. Adjustable volume control of the loudspeakers is very recommended. The TOSSA runs on a stand-alone computer and on a network server. N.B.: During executing the TOSSA all other programs should preferably not be running (think

of screensavers, anti-virus programs, email programs and hard disk-pause programs). When you still have some trouble in running TOSSA you can email F. Kovács: [email protected] or use the contact form on www.pyramidproductions.nl. Furthermore, when running on a laptop please be sure NOT to use the battery. Instead, use the power cord. A battery run can seriously interfere with the smooth running of the TOSSA. The same goes for every power-saving mode!

The Test of Sustained Selective Attention consists of a short instructions part, a short practice part and the 8-minutes test itself. One has to listen to 240 groups of 2, 3, or 4 beeps. Whenever a group of 3 beeps sounds, the spacebar has to be pushed as quickly as possible. The target then is 3 beeps, 2 and 4 beeps are the distracters. The intensity of concentration required in this test is influenced by three factors:

1. The target (3 beeps) is quite similar to 1 distracter (4 beeps); 2. The inter stimulus interval (ISI) varies: initially this time is gradually reduced with 20 ms per

stimulus. With the first 60 stimuli there is a barely noticeable increase in speed. After 60 seconds, this speed becomes reversed and it is reduced gradually with 20 ms per stimulus. This increasing and decreasing the ISI is repeated 3 times (4x 60 stimuli is a total of 240 stimuli in 8 minutes and 16 seconds).

3. The test runs for 8 minutes and 16 seconds which requires some sustained concentration. Administering the TOSSA varies from 10 to 12 minutes, partly dependent on the speed of a patient. N.B.: The exact total time printed on the printout should be 494.6 seconds (see Appendix I). Any

changes in this time being larger than 5 seconds should be considered wrong and then the TOSSA results can not be interpreted reliably anymore using the current norm groups. On modern computers this slowing would be extremely rare.

After the test has ended, the computer registers the following in an ASCII text file (see for an example Appendix I, page 58): - the stimuli and all reactions on these stimuli. These reactions comprise of 5 types:

1. h=hits: correctly reacted to a target stimulus (a group of 3 beeps). 2. f=false: falsely reacted to 2 or 4 beeps.

mailto:[email protected]

http://www.pyramidproductions.nl/


_______________________________________________________________________________________4

_________________________________________________________________________________________


3. v=premature reactions to 2, 3, or 4, before the stimulus was completely presented. 4. t=slow reaction on a target (3) so that in effect a reaction is registered to the next

stimulus. 5. o=omission: a skipped or undetected target on which there was no reaction at all.

- a summary of the data per block. A block consists of 120 stimuli in which 60 stimuli are repeated once and the inter stimulus interval varies from 440 to 1640 milliseconds. - 13 scores c.q. indices of which the first 7 are considered to be the most important ones: CS: Concentration Strength: the intensity of the concentration calculated via this formula:

(DS*RIS)/100. The range of CS is: 0 - 100%. This is seen as the most important TOSSA index.

DS: Detection Strength= number of hits / total number of targets presented * 100% (range:0-100%). Maximum number of targets is: 80 (20 on each of four rows).

RIS: Response Inhibition Strength: the strength with which the impulse to react on a distracter

(2 or 4 beeps) can be repressed. Formula: ((240-number of faults-number of premature \ reactions)/240)*100. Range: 0-100%. In this index there are two aspects of selective

attention. On the one hand the concept of distractability in which someone will loose his focus when distracted by other (irrelevant) stimuli. On the other hand there is the concept of impulsivity: in fact a behavioural component of distractability. Being distracted means losing focus and detection strength drops. However, it can also mean that someone gets more passive and starts reacting less. Or someone gets more active and starts reacting to irrelevant stimuli. More about this on the pages on interpreting the TOSSA scores and profiles.

SADS: The influence of speed on the DS-index. Formula: ((DSS-DST)/DST)*100, expressed in percentages. The change in DS in the fastest part of the test is expressed against the slowest part. Range: -100 to 3900. This last score is theoretically possible if one half of the test (DSS) is done 100% and the other half (DST) is done for 0% or 1%. An example: a SADS of 50% means that someone improves his score with 50% compared to his score in the slowest half of the test. A negative SADS of -10% means that someone lowers its score by 10% compared with his performance in the slowest half. This index largely represents how much learning effects within the TOSSA counteract the factor of increasing fatigue. Fatigue increases as the test continues. It can be expressed in a lesser effort but also in continuing to react, although often incorrectly, increasing the numbers of errors. LADS: The influence of the Length of presentation on the DS-index. Formula is:

((DSB2-DSB1)/DSB1)*100. The changes in DS are expressed against the DS in the first block. Range: -100 to 3900 (see remark made about range in SADS).

SARIS: The influence of speed on the RIS. Formula is: ((RISS-RIST)/RIST)*100, range: -100 to 11900. This last number is extremely rare and can only exist when RIST=(1/120)*100%=0.833% and RISS=100%. This index expresses how much someone is influenced on his ability to control himself when he is put under (time) pressure. Someone can become more passive and will not react correctly anymore on targets. Someone can become more active but starts to react (incorrectly) on the distractors (increased distractability and impulsivity).

LARIS: The influence of the length of the test on the RIS. The formula is: ((RISB2- RISB1/RISB1)*100%, range: -100 to 11900. This index expresses how much the learning effects are counteracted by fatigue. It can be that under pressure someone will react more impulsively on the irrelevant stimuli.

DST: DS in the slowest part of the test. Range: 0-100%. DSS: DS in the fastest part of the test. The test is divided in 2 halves: the first 30

stimuli are in the slow part and the second 30 stimuli are in the fastest part of the test. For each part the DS is calculated. Range: 0-100%.

DSB1: DS in block 1. Range: 0-100%. The test is divided in 2 identical blocks: each block


_______________________________________________________________________________________5

_________________________________________________________________________________________


consists of 120 stimuli: 40 target stimuli in which the ISI increases and decreases. Block 1 is the first block, block 2 the second one.

DSB2: DS in block 2. Range: 0-100%. RIST: RIS in the slowest part of the test. RISS: RIS in the fastest part of the test. RISB1: see RIS but this is for block1. RISB2: identical but for block 2. CST: CS in the slowest part of the test. Range: 0-100% CSS: CS in the fastest part of the test. Range: 0-100%. CSB1: CS in block1 CSB2: CS in block2 SACS: The influence of speed on the CS-index. Formula: ((CSS-CST)/CST)*100, expressed in percentages. The change in CS in the fastest part of the test is expressed against the slowest part. Range: -100 to 3900. See further explanation at SADS. LACS: The influence of the Length of presentation on the CS-index. Formula is:

((CSB2-CSB1)/CSB1)*100. The changes in CS are expressed against the CS in the first block. Range: -100 to 3900 (see remark made about range in SADS).

1.2. Administering the TOSSA: procedures and instructions

______________________________________________________ Place the patient at about 70 cm in front of the computer screen and the keyboard in front of him or her. N.B.: the sound of the beeps should be tested before initiating the TOSSA. Adjust the volume of

the loudspeakers so that the patient can hear the beeps clearly without becoming annoyed.

1.2.1. Starting the TOSSA Run the TOSSA by double clicking on the file TOSSA.EXE in Windows Explorer. Or you can use the Start Menu shortcut TOSSA or Test of Sustained Selective Attention. On screen the colored intro screen appears of the TOSSA (Figure 1).

Figure 1. Intro screen which disappears in several seconds


_______________________________________________________________________________________6

_________________________________________________________________________________________


The menu of the TOSSA is very self-explanatory (Figure 2):

Figure 2. Menu of the TOSSA First a new patient has to be made. Click on the most left icon: all fields should be filled in: name, birth date, education (code see page 67) and diagnosis. Please click on Start New test and the TOSSA starts. First the screen will go black and the instructions are shown. Below you will find these instructions that have to be read for the patient.

1.2.2. Instructions "In this test you will hear groups of 2, 3, or 4 beeps. You will have to react only on groups of 3 beeps. This is how the beeps sound [PRESS on 2, 3 or 4]. Notice: Only press the SPACEBAR whenever you hear 3 beeps. Before we’ll start the practise round I will first let you hear the beeps. Please tell me how MANY beeps you hear now" Press on number 3 of the keyboard, then 2, then again on 3, then 4. Do this randomly as to prevent guessing. You have to make certain the patient can hear 3 or 4 or 2 beeps clearly and reliably. Whenever there are doubts, please repeat this process again. Then you can start the practice round. Please make sure that the patient can hear reliably the difference between the different groups of beeps. Whenever this is not the case and the practice round shows a majority of errors then the TOSSA can not be reliably administered due to auditory discrimination problems. " I will let you hear the group of 3 beeps a couple of times. " Press a number of times on the 3, relatively quickly, a maximum of 5 times. It is important the patient has a feeling of the ‘melody’ of these 3 beeps. Emphasize the melody and discourage explicitly to count the beeps: " Counting is not wise. You can better listen to the melody in the 3 beeps… it sounds as if someone is knocking on your door. " Give 3 knock on the table as an example. Press 3 once again.


_______________________________________________________________________________________7

_________________________________________________________________________________________


" Shall we just rehearse? Hold your finger here, just below the spacebar, resting on the keyboard. Please make sure your hand lies comfortable and your whole arm is resting quietly" N.B.: It is very important that the finger of the patient lies very close to the spacebar so any reaction can be quick. When this reaction finger is resting on the right spot, the Rehearse button can be pressed, either with the mouse cursor or by pressing the TAB key. After this rehearsal the results are shown on screen (Figure 3). You can explain these to the patient. Whenever these results clearly show that the patient can not discriminate reliably between 3 beeps and the other beeps, the test can not be done.

Figure 3. Screen after first rehearsal with results (in version 4 this screen is darker). N.B.: With 4 or less hits please repeat this rehearsal once more. “ Shall we do this rehearsal once again? But first I’ll let you hear the 3 beeps again.”. press again 2-3 times on 3, with an interval of at least 2 seconds between each press. “Please just press the spacebar whenever you hear this 3 beeps.” The instruction is repeated again to ensure that the patient has fully understood it. Then press the Rehearse Button again. During this second rehearsal it is allowed to help and stimulate the patient to detect the 3 beeps by giving feedback about his response. N.B.: Rehearsal will be repeated only once. When the score still remains less than 4 hits correct the whole test will be administered unless:

1. The patiënt himself is not motivated anymore to do the test; 2. there are clear indications that the bad performance was due to a hearing deficit or an

auditory discriminative disorder (=severe difficulties to reliably discriminate between 3 and 4 beeps). For example, when there 5 out of 7 incorrect responses on 4 ánd correct responses on 3.

" I want you to do the real test which will take 8 minutes. Please try to maintain your concentration. You will notice this will not be easy but just try to do your best. Never panic, just try to do your best, nothing more, nothing less. " It is important to stress that patients can only do their best and that they don’t have to panic. " I will leave the room during the test so you can do it without any disturbances." In clinical practice it was shown that many people started to make comments to the tester. That’s the main reason the test has to be administered without anybody around. However, when starting the test care has to be taken that the patient really detects a target 3. When the first target is correctly


_______________________________________________________________________________________8

_________________________________________________________________________________________


pressed, a verbal feedback can be given (“good”) and then you can leave the room quietly. Whenever the patient incorrectly presses the space bar, direct feedback can be given (“no, please press only on 3’s”). Then you can leave the room. More help is absolutely forbidden, even when someone presses incorrectly. Be sure when leaving the room that no mobile phone, telephone or fax is standby!). In about 8 minutes you can come back. When the TOSSA ends the Remarks screen will be displayed (see Figure 4).

Figure 4. The remarks screen with the Close button.

1.2.3. Storing the TOSSA results: manual setup and lay-out of the report fully adjustable The data will be stored in a file ‘Report.txt in the default directory C:\Users\Computername\AppData\Roaming\Tossa. Actually there are 3 other files there: Data.txt (only for Excel), graph1.jpg, graph2.jpg. These separate files are used to generate the final test report. The storage default directory can be changed easily by clicking in the menu on Settings (4th from the right). Here you can choose for a new data directory. Every administration will be stored as Patiënt1, patiënt2, etc. In the menu you can also find a list of patients to which the TOSSA was administered. See Figure 5.

Figure 5. List of patients to which the TOSSA was administered


_______________________________________________________________________________________9

_________________________________________________________________________________________


When double clicking on the patient name, the administration times will be displayed. (Figure 6).

Figure 6. List with administration dates and times of the TOSSA Double clicking on the date and the test report is shown (Figure 7). This test report can also be found in the Data storage directory.

Figure 7. Report and test data. This can both be stored in a RTF and PDF format


_______________________________________________________________________________________10

_________________________________________________________________________________________


The 2 graphs below will be placed automatically in the report.


_______________________________________________________________________________________11

_________________________________________________________________________________________


The report can be saved in both a RTF- and a PDF-format with the Export Icon (white page with green plus sign). The RTF-file can be stored in Word and adjusted using your own text and logo.

1.2.4. Tips for an optimal administration of the TOSSA To optimize testing conditions:

• With every neuropsychological test it remains important to optimize the testing conditions. Every reassurance, stimulation or extra explanation can be given to a nervous patient. What you may NOT convey to the patient is that the TOSSA has a varying speed! You may emphasize that whenever the test is becoming difficult the patient may let go and try to focus later on. The patient should not panic!

Avoid any disturbances during the test:

• Please take care of any standby noisy equipment like mobile phones or other telephones or clocks. Also take care no noises are heard through an open window.

Adjust the volume of the loudspeakers to hear the beeps clearly enough:

• Please see to it that the loudness of the beeps is comfortable and efficient. Aborting the test at any moment:

• The TOSSA can be aborted at any moment by pressing the ESCape key on the keyboard. N.B.: it will take some 5 seconds before the test closes. So during 5 seconds you will see the information where the results are stored on your computer. Printing the results: The TOSSA itself has no print facility. Data are stored in text files with the extension

nameTOSSA.txt. For example, patient KOVACS will be stored as KovacsTOSSA.txt in the directory you have set in the DataStorage.txt file or in the installation map where the TOSSA is installed in. You can open these text files in any word processor and print them.


_______________________________________________________________________________________12

_________________________________________________________________________________________


1.2.5. Interpreting the TOSSA results In the example below a printout of the TOSSA-results is shown. Each step shows how to interpret these results. Test of Sustained Selective Attention for Windows Version 3.0 Build 1

Surname: kovacs Date of birth: 28-08-1964

Age: 47 Date of test: 29-6-2012 16:45

Educational code: 7 Sex: m Diagnosis: healthy

Remarks:

Practice:

stim 23332423444344242323

1 ooh f h h f f h h

2 hhh h h h h

Longest isi: 1640 ms Shortest isi: 440 ms

123456789012345678901234567890123456789012345678901234567890

stim 242343443432242342323223342342342234234233324234443442423234

1 h h h h h o h oo fh h o h hoh o h v ov

h h h h h h h hh h h h h hvo fov vo h o

2 o h o o h h h hh h hf h ov hoo h ov f o o

fv h o o f o h h hof h o fo h ooo o h o of

block h o f v t

1 27 11 2 5 0

2 18 21 7 3 0

Tot: 45 32 9 8 0

Total errors on 2: 4

Total errors on 4: 5

Total prematures on 4: 5

Compared to Healthy controls N=224:

Concentration Strength (CS): 52.3 moderate concentration deficit: <d1, <1p

Detection Strength (DS): 56.3 moderate loss of focus and deficit: <d1, <1p

ResponsInhibStrength (RIS): 92.9 very obvious impuls control deficit: <d1, <5p

Compared to Right-Stroke patients N=297: decile 5

Compared to Left-Stroke patients N=284: decile 4

Compared to severe Traumatic Brain Injury patients N=145: decile 4

Compared to Other neurological patients N=293: decile 3

Compared to WAD type II patients N=82: decile 1

Profile suggests Suboptimal Effort when this is a healthy person

DSSratio is: 0.55 DSLratio is: 0.67

SADS (influence Speed on DS): -44.8

serious influence of speed on focus, deficit: <d1, <1p

LADS (invloed Lengte test op DS): -33.3

serious influence of length on focus: <d1, <1p

SARIS (influence Snelheidsverhoging op RIS): -6.1

moderate influence of speed on impulse control: <d1, <1p

LARIS (influence Snelheidsverhoging op RIS): -2.7

obvious influence of length on impulse control: d1

DST (DS in slowest half: 72.5

DSS (DS in fastest half: 40.0

DSblock1 (DS in 1st half: 67.5

DSblock2 (DS in 2nd half: 45.0

RIST (RIS in slowest half): 95.8

RISS (RIS in fastest half): 90.0

RISblock1 (RIS in 1st half): 94.2

RISblock2 (RIS in 2nd half): 91.7

CST (CS in slowest half): 69.5

CSS (CS in fastest half): 36.0

CSblock1 (CS in 1e blok): 63.6

CSblock2 (CS in 2e blok): 41.3

SACS (influence of speed on CS): -48.2

LACS (influence of length on CS): -35.1

1

2

3

4

5

6

7


_______________________________________________________________________________________13

_________________________________________________________________________________________


Here the Concentration Strength (CS) index is shown and its significance. In Tables III to XXIV you can find how this CS index deviates from the mean of the specific norm groups. N.B.: whenever TOSSA shows a concentration disorder (like in this example), there is a serious deficit in concentration! Do not be misled by labels such as ‘slight’ or ‘moderate’. These are only made to differentiate between deficits being very severe and less severe. The terminology ‘deficit’ is based on a deviation of 2 standard deviations of the healthy control group mean (about 5% or less of the scores of healthy controls). The terminology ‘problem’ refers to concentration difficulties present but not in such severity that there is a 2 SD deviation from the healthy group mean. Here you can compare the CS-score with 5 different norm groups. In this printout this CS-score is sufficient when compared to the right-Stroke patients (decile 5). And it is a bit worse when compared to other neurological patients (decile 3). A low CS-score can be caused by a low arousal, often seen in many omissions on 3’s. Another cause can be a much too large distractibility where the patient has reacted too much on distracters (like 4 beeps). Also a high impulsivity can be a factor to reduce the CS-score. That is when there are far too many premature reactions to 4 beeps (in which 3 beeps can be heard). Finally, a motivational problem or apathy can lead to no reactions at all, thereby reducing the CS-score. To interpret a CS score correctly the next steps are necessary. A CS-score is the most sensitive TOSSA index for a concentration problem because it represents two aspects of concentration: 1. Detection strength (DS): how far correct reactions on 3 beeps were registered? and 2. The Response Inhibition Strength (RIS) that represents how many incorrect reactions were registered on the distractors 2 or 4 beeps and on the 3 beeps. In this example, the CS is 52.3%. The question now is: is this because distraction and high (time) pressure has increased the number of omissions? Or does the pressure has increased a tendency to react more impulsively? Five other indices might answer these questions. The influence of the increasing speed on the DS-index is shown here (SADS). In this printout there is a deterioration of 44.8% compared to the slowest part of the TOSSA, way below the 1st percentile. Whenever there is a large SADS you will first have to check if the DST score is reasonable (here it’s not). When this is indeed the case, then increasing speed (time pressure) has played a major role in reducing the DS-score. Mental slowness is then a valid conclusion. However, when the DST score is far below normal as well (as in this example), not only mental slowness plays a role but then there is a structural concentration deficit as well. The RIS-index (Respons Inhibition Strength) displays the strength of control over the impulse to react to distracters. Responding to a distracter of 4 beeps is especially triggered because the target 3 beeps can be heard in these 4 beeps. So whenever a patient has too many premature reactions to a 4 distracter (v) this signifies serious inhibition or impulsivity problems (such as here). Reacting on the 2-beeps distracter has probably different causes than just impulsivity. Just as a reaction on the full 4 beeps distracter. Both reactions can probably seen as a form of distractibility: loosing the focus on the 3-beeps target by external pressure such as time pressure or fatigue. When RIS is becoming very low due to time pressure (high negative value of SARIS), this can mean an active form of distractability where someone starts acting impulsively whenever his attention is weakening. With LARIS you can see how fatigue and test length can influence this impulsivity: the more negative this score the less someone can control his impulsivity due to fatigue. Please note that the TOSSA does not trigger impulsive reactions very much. In Table IV the healthy group scores 98.2 on RIS (median), the cut-off is 90.2%. Comparisons of the RIS with other patient groups will tell you how extreme this score deviates. Whenever there are more premature reactions on 3’s within a group of 4 beeps, this certainly tends to show impulsivity (like here). Whenever there are more errors (f’s) then someone has lost his focus on the right target. That has more to do with distractability. The LADS index shows the influence of the test duration on the DS-score. In this example there is a reduction of 33.3% due to the increasing test length. That’s extreme, below the 1st percentile. Most healthy people have such an improvement: they perform better when the test continues, probably due to a slight learning effect. Whenever the LADS is very low (-10%), either fatigue plays a role or diminishing motivation. This last factor can easily be seen when there are no reactions at all in the last minutes of the TOSSA. Here you can see that this is indeed the case.

2

1

3

4

5

6


_______________________________________________________________________________________14

_________________________________________________________________________________________


TOSSA automatically calculates several profiles to ease the interpretation of this test. The following profiles have been constructed out of the data in the healthy controls group and the right stroke patients group. Passivity Several profiles are largely centered around passivity, reacting less due to (time)pressure: Profile Mental Slowness: when the fastest part of the test is considerably less than the slowest part, then this conclusion is highly probable. Profile Enhanced Fatigability: when performance in the 2nd half drops considerably compared to the 1st half, this suggests a fatigability that can be triggered easily. Profile Endogenous Arousal problems: when the number of reactions strongly drops in both the fastest and 2nd half of the test and the patient seems to become more lethargic (not impulsive), then the conclusion seems valid that the own endogenous arousal system is not capable to build up enough energy to withstand severe task pressures. This phenomenon of withdrawal under pressure has probably a lot to do with motivational problems and can be seen for example in depressed patients. Further research may corroborate this hypothesis. Impulsivity or increased reactivity Several other profiles are mostly centered around a less correct active reacting on the test, usually considered a sign of impulsivity: Profile Structural Impulsivity: a form of many impulsive reactions in all conditions with some pressure, be it under time pressure or due to the long duration of the test. There seems to be no relationship between reacting impulsively and the test conditions. Profile Conditional Impulsivity: here there IS a relationship between the test conditions and the impulsive reactions. Especially under time pressure, patients are prone to react incorrectly and impulsively. However, the profile is only valid whenever there is a sufficiently large difference between the slowest and fastest half of the test. Reduced effort or aggravation Three other profiles are centered around the concept of reduced effort, usually represented in the literature with the term ‘malingering’ or ‘aggravation’. It is extremely difficult to ascertain whether such worsening of one’s own performance is done deliberately and consciously or subconsciously. Therefore a word of caution: the profiles are being calculated automatically on the basis of especially rare reactions, but it still requires a skilled investigator to wage all possible other evidence to finally conclude something about suspected reduced effort. Profile Auditory perception problems or misunderstood instructions: here reactions are found that are extremely rare in healthy controls and in the right-stroke patients group. Often this has a lot to do with not being able to discriminate between 3 or 4 beeps. That should have been clear at the practice session and it can always be asked áfter the TOSSA has been done. Profile Suboptimal Effort: on the basis of several parameters which are extremely rare in the neurological patients group (N=324), see Tables XXXIV and XXXV, page 53, this conclusion is drawn. But again: these profiles signal something. Real symptom validity tests and other data have to be consulted extensively before such firm conclusions about reduced effort or malingering can be made. For example, it is always advisable to look at the raw data. What is rare are many errors on 2 beeps. Especially when there are lesser errors on the 4 beeps or when there are quite a few correct reactions on 3’s in the faster part of the test. Also a pattern of consistent fluctuating correct reactions on 3’s is very suspicious because such patterns cost a lot of skill and energy and are almost non-existent in neurological patients. .

7


_______________________________________________________________________________________15

_________________________________________________________________________________________


2. Theoretical background ______________________________________________________

The Test of Sustained Selective Attention (TOSSA) is based on several vigilance tasks in which one has to react to 1 target amidst more distracter stimuli during a relatively long time period. This paradigm actually is based on the continous performance task developed by Rosvold et al. (1956). In Shiffrin and Schneider’s model of information processing (1977) this form of attention is called 'focused attention'. Elsewhere it is also referred to as selective attention. However, when such a form of focused attention has to be maintained for a longer time period, it can also be seen as a form of ‘sustained attention'. Researchers are still debating about how long such a time period should be to consider it sustained attention. Current insights on attention change the much used terminology of selective or divided attention, suggesting that it is inconstructive. The heuristic model of Tim Shallice (1982 and 1988) is used instead wherein a Supervisory Attentional System (SAS) is seen as central (Norman and Shallice, 1986). This concept can be compared or even equated with the concept of the Central Executive of Alan Baddeley (1986) and the Attention Director of Shiffrin and Schneider (1977). The Supervisory Attention Control (SAC) system is a postulated process for the control of non-routine information processes. Such a control is considered to be largely conscious, although this word usually is avoided in scientific literature. But studying the operationalisation of ‘controlled processing’ one cannot but conclude that this certainly is a conscious form of processing. This is in sharp contrast to the automatic processing of routine information processing units called modules, which takes places unconsciously (although this terminology is carefully avoided as well). The model of Shallice can place several attention tasks on a continuum of using more or less Supervisory Attentional Control and therefore is a more useful model to compare attention tests. Especially when the alternative is to try to differentiate attention tests on the basis of concepts like ‘divided, selective or sustained attention’. Extending the work of Shallice, Miller and Cohen (2001) have presented an influential model centered around the concept of ‘cognitive control’. In fact, when studying their article it is surprisingly similar to the concept of the Supervisory Attentional Control or the Attention Director in Shiffrin and Schneider’s model (1977). Unfortunately, this is not explicitly mentioned in their article, but again, when studying their definitions it is extremely hard to find any differences in these concepts. The elegance of Miller and Cohen’s model however is that ‘cognitive control’ is defined parsimoniously: ‘the active maintenance of neural action patterns that represents goals and their means’. By keeping goals and means actively ‘online’ (in working memory), it is possible to regulate and direct the largely automatized information processed that takes place in the brain so that goal-directed behavior becomes possible. For a schematic explanation see Figure 8.

In another definition of ‘cognitive control’ Miller and Cohen clearly relate this concept to goal-directed behavior and this surely is the exact same function of the Supervisory Attentional System: “the internal representation, maintenance, and updating of context information in the service of exerting control over thoughts and behavior. …We define context as any task-relevant information that is internally represented in such a form that it can bias processing in the pathways responsible for task performance” (Braver en Barch, 2002). Very elegantly this definition explains difficult concepts like inhibition, working memory and attention as simply being aspects of cognitive control (p. 811) without the need to postulate these as independent and differing concepts: “…the context processing functions of our model demonstrate how a single underlying mechanism, operating under different task conditions, might subserve three cognitive functions that are often treated as independent—attention (selection and support of task-relevant information for processing), active memory (on-line maintenance of such information), and inhibition (suppression of task-irrelevant information)”. Selective attention is considered here as a process that searches for task relevant information and maintains it in an active state, in working memory, which is located in the dorsolateral prefrontal cortex (Braver, 2007).

In this ‘goal-maintenance’- model of Miller and Cohen many tasks can be explained quite easily such as the Stroop tasks, the Wisconsin Card Sorting Task but also several attention tasks like the continuous performance tests such as TOSSA. In the TOSSA one goal has to be actively remembered during interference of distracters: ‘press only on 3 beeps’. The stronger this goal is activated and maintained in working memory, the better the performance on the TOSSA. But at the mean time another rule or goal should be active as well: not pressing on 3 beeps in a group of 4 beeps. This requires active inhibition of impulsive responding. When the activation of the first goal (3 beeps) is lost (due to fatigue or distraction), either no reactions at all or false reactions to distracters are possible.


_______________________________________________________________________________________16

_________________________________________________________________________________________


Interpreting whether there is a reduced response inhibition or distractibility when someone reacts too often on groups of 2 or 4 beeps, remains unclear. The theoretical attention model of Miller and Cohen states that selective attention means that a target has to be held in ‘working memory’, despite distractions and time pressure. This should cost quite some metabolic energy and is only to maintained temporarily. When the pressure is rising – in the TOSSA especially due to time pressure – more energy should be spent on focusing on the instruction and the target in working memory. Reacting more quickly takes even more metabolic energy and increases stress (anxiety). In the TOSSA someone could react to this stress in several ways:

1. correct and active: reducing reaction times and more repeating of the target in

working memory. The so-called exogenous component of attention is all right: attention is especially drawn by the increasing task demands. This in fact, is nothing else than the triggering of the arousal-attention system. The phasic alertness/arousal is activated by increasing task demands. Lower brain structures like the reticular formation (ARAS), locus coeruleus, hypothalamus and amygdala are involved. Not by accident also the same regions having to do with the stress- (flight or fight) reaction. The most important neurotransmitter here is noradrenaline. This also explains the phenomenon that some patients are in fact improving when the pressure increases. They show the tendency to commit more omission errors in the slower (duller) half of the TOSSA.

2. Incorrect and inactive: with increasing pressure someone can decide (consciously or unconsciously) to be more careful and to react less (RISS is high but DSS is low). Another option would be that time pressure is overwhelming someone, thereby reducing his motivation to continue and lowering his (phasic) alertness. Here a more lethargic pattern will be seen: during time pressure very few to no reactions (a lot of omissions, low DSS) but also a high RIS. Often you will see that such a ‘motivational blow’ is extended in the second half of the test (with a lower LADS). A factor like fatigue can join in as well. Other explanations for a high negative SADS can be mental slowness and becoming overwhelmed.

3. Incorrect and active: with increasing pressure someone can decide to react more often. Here, the increasing task demands trigger the phasic alertness system in such a way that someone is unable to control this arousal anymore. More impulsive reactions can then be found. Reacting correctly in the TOSSA requires a combination of executive attention (probably originating from the dorsolateral prefrontal cortex and anterior cingulate cortex) and the perceptual attention system (right temporal-parietal lobe, see Posner and Peterson’s model, 1990). Especially the executive attention system has a role in regulating impulsive reactions. When SARIS is high-negative this means that this regulation hasn’t worked properly. Closer inspection can reveal whether there was more impulsivity (DSS high but RIS low) or more distractibility (RIS low and DSS low).

To help with the interpretation of the TOSSA results, several profiles will be calculated automatically and suggested. N.B.: everyone surely knows that firm conclusions should not be drawn based on only one neuropsychological test. Other test results, a good interview and observations are important as well to arrive at the right conclusions. Within the ‘goal-maintenance’-model one can place the TOSSA quite well, just as the Stroop task, the Trail Making Test or the Paced Auditory Serial Addition Task (PASAT). However, it remains very difficult to quantify the amount of selective attention needed in each test. Subjectively it seems that the Stroop is requiring more focus and selective attention than the Trail Making Test. However, both tests last only around 4 minutes or less. That is certainly not the case in the TOSSA in which for over 8 minutes a focus has to be maintained. A very demanding test is the PASAT because it lasts long (over 8 minutes) and a very strong response tendency has to be inhibited constantly (namely, adding numbers sequentially). The problem with the PASAT is that even a lot of healthy people do have serious problems with this test.


_______________________________________________________________________________________17

_________________________________________________________________________________________


Figure 8. The Goal-maintenance Model of Miller and Cohen (2001). Another well-known model of attention is that of Kinsella (1998) who recognizes that there can not be one dimensional definition of attention because it is a multi-dimensional concept with the corresponding different neuronal networks. Heavily based on the work of Posner and Peterson (1990) she summarizes 3 components of attention: Orienting: a largely automatic reaction to a stimulus (phasic alertness). Three phases have to be run

though: disengagement of the focus on an old stimulus, a directional movement (where to go with the focus) and third: re-engagement to the new stimulus. The posterior attention network is probably located in the right posterior parietal lobe, pulvinar thalamus and reticular nuclei and the colliculus superior. Disturbances in this system result in object recognition problems, unilateral inattention and slowed reactions to external stimuli.

Selection: this system strongly resembles the SAC-model of Shallice. It is largely controlled (and conscious) information processing that regulates where attentional resources are allocated. Neurologically, the middle prefrontal cortex and the basal ganglia are supposed to mediate this system. Although this anterior attentional system is heavily connected to the posterior system, they can function quite independently from each other. Disturbances in the anterior system can result in divided attention problems, problems in neglecting interfering stimuli and the adequate responding to new tasks (flexibility).

Sustained attention: The ability to concentrate and maintaining this focus for longer time periods. Possibly the locus coeruleus and the right hemisphere are involved in this. Disturbances can result in a shorter attention span and easily developing fatigue.

Although differentiated academically, all three networks closely operate together in each attention test. In the TOSSA, orienting is necessary to react quickly to every target. The selection system is required to react correctly to a target and nót to a distracter (inhibition). The sustained system is needed to maintain the focus level during 8 minutes.


_______________________________________________________________________________________18

_________________________________________________________________________________________


These 3 attention components – taking initiative, not getting distracted and sustained attention – are assessed by a new attention scale MARS (Moss Attention Rating Scale) developed by J. White et al. (2003). The TOSSA cannot be easily compared with existing attention tests. Firstly, the TOSSA is an auditory test. Secondly, it is a dynamic test with a varying speed of stimulus presentation to dynamically challenge the attention system. The only comparable tests are, of course, the other continuous performance tests (CPTs). Four commercially available CPTs were reviewed in Riccio, Reynolds, Lowe and Moore (2002): Conners’ CPT, TOVA (Test of Variables of Attention), Gordon Diagnostic System (GDS), and IVA (Integrated Visual and Auditory CPT). Mostly, these 4 CPTs are being used to detect Attention Deficit Hyperactivity Disorders and consequentially have been administered mostly to children. Unfortunately, no norms are available for groups of neurological patients such as with the TOSSA test. It would be very interesting to compare the TOVA-A (auditory version of the Test of Variables of Attention, Greenberg, 1999) with the TOSSA. There are only two stimuli in this TOVA: two short tones, one the target, the other the distracter. One has to press as quickly as possible on a target tone. One half of this test (duration 21.6 minutes!) has few targets in a ratio of 1:3.5 (target- non target). The other half has much more targets to detect impulsivity in a ratio of 3.5:1. This task is not only very long for a practical and clinically useful test, even a lot of healthy people have difficulties with this (boring) test. Therefore, the scores do follow a normal distribution which is good in a statistical sense. But then a large number of norm groups or healthy people are needed. Currently, more than 2000 people have been tested with the TOVA, ranging from 6 to 80 years. A rather strange fact is that in European neuropsychology CPTs are barely used in research and in clinics. Other, quite different tests like the Stroop Color Word test, the Trail Making Test and the Paced Auditory Serial Addition Task, are used more frequently in populations with brain injury. Due to different versions of such tests no convincing research databases with consistent data have been gathered yet. TOSSA, which is a very standardized test due to its computer format, has been developed to change this sorry state of affairs.

As stated above, CPTs cannot be easily compared with other attention tests (Riccio et al, 2001, p. 110). They measure some unique aspects of executive attention other tests do not detect. Furthermore, the correlation between CPTs and behavioral observation scales of attention are very reasonable (.40 - .60, Riccio et al, 2001, p. 111). Most correlations between CPTs and standard attention tests are however very modest: between .30 and .50. Most studies have used only small numbers of patients (Riccio et al, 2001, p. 116-117). Finally, much more research is needed in using CPTs with more adults because most data are about children.


_______________________________________________________________________________________19

_________________________________________________________________________________________


3. Norm research and psychometric characteristics ________________________________________________________ 3.1. Research in norming the TOSSA The norm data for the TOSSA have been gathered rather conveniently over more than 14 years. Since 1994, data of the TOSSA were systematically collected in consecutively admitted patients to a small rehabilitation clinic in the Netherlands. No planned and randomly stratified data collection has taken place. At the end (of 2009), a group of 224 healthy controls and 1101 patients had been administered the TOSSA. The healthy group of persons (N=224) without any form of brain damage (now and in the past) consisted of 5 groups:

1. a group of 49 students of the school for dental technicians in Nieuwegein in the Netherlands. TOSSA was administered in September 1994.

2. a group of 53 volunteers in Voorhout and the surrounding region who had reacted to local newspaper ads to participate in a study for norming the TOSSA (1997-1999).

3. a group of family members and employees of the rehabilitation centre (n=32). 4. a small group of high school students who participated in a biology study and were tested

with the TOSSA (in 2006-2007, n=9). 5. a relatively large group of volunteers recruited via Internet who wanted to know more about

their attention capacities (2004-2006, n=81). This last (internet) group requires some explanation. Volunteers could download a special edition of the TOSSA which had to be run at their own homes. The TOSSA installed itself and all instructions were shown on the computer screen. The test could be run only once, to prevent any test-retest effects (or getting used to the test). The coded results were uninterpretable and the volunteers were asked to send these results to the author. Via email questions were asked about in what conditions the TOSSA was administered at their homes, and about their medical history focusing on the question whether they have had any form of brain injury in their past or did have any medication during taking the TOSSA. Only when there was adequate certainty about all these conditions, the test results were collected and used in statistical analyses. Less than 5% of these collected results had to be deleted due to uncertainty about following the correct instructions or the healthy status of the volunteer. Furthermore, this group of healthy internet volunteers was statistically compared to the other healthy volunteers not recruited via Internet (n=144). On the indices DSB1, SADS en SARIS there was a significant difference between the internet group (n=81) and the healthy controls (n=143), respectively p=.03, p=.05 and p=.04. Of the demographic data (age, sex, education) only education differed significantly (Internet group vs Controls: mean 6.0 vs 5.5: p=.001 (Mann-Whitney U)). Because of these very small differences, it was decided to join both groups to one larger group of healthy controls.

Another statistical analysis was performed to find out whether there had been any change in the healthy population over all those 15 years (1994 till 2009). The group of healthy controls collected before 2001 (n=120) was compared to the healthy controls group collected after 2001 (n=104). On only one index (DSB1) there was a small but significant difference. In other words, the group after 2001 was slightly better on the DSB1 index (difference of 2.2%) than the older group before 2001 (Mann-Whitney U test). In depth analyses revealed that these differences were largely due to the Internet group. Probably, they were more motivated and slightly more educated (6.0 vs 5.4, p<.001). In a later discussion it is shown that education has a weak but statistically significant linear correlation with the TOSSA indices: CS variable (Spearman’s rho= .26, Kendall’s tau_b=.20, both p=<.001), de DS (rho=.24, tau_b=.18), de RIS (rho= .23, tau_b=.18, p<.001). The higher education is, the better the TOSSA will be (see Figure 9). However, this correlation is considered too small to correct for it. The same goes for the very weak correlation between age and CS (Pearson’s R=.11, n.s.), see Figure 10. Further research with more healthy controls should explore this more.


_______________________________________________________________________________________20

_________________________________________________________________________________________


Figure 9. The weak correlation between Education and the CS index within the healthy controls

group n=224

Figure 10. The non-significant relationship between age and the CS index within the healthy controls

group n=224)


_______________________________________________________________________________________21

_________________________________________________________________________________________


HEALTHY controls group (N=224):

Figure 11. Age distribution in the HEALTHY controls group (N=224); mean: 33.6 yr, range 15-93 yr, SD=15.8 yr

Table I. Distribution of Sex in the HEALTHY controls group (N=224)

Sex: 1 =male ; 2 =female

103 46,0 46,0 46,0

121 54,0 54,0 100,0

224 100,0 100,0

1,00

2,00

Total

Valid

Frequency Percent Valid Percent

Cumulative

Percent

Table II. Distribution of Education* in the HEALTHY controls group (N=224); mean 5.7, range 3-7, SD=.95

*: according to Verhage (1964), see Appendix I page 58

28

94

46

21 11 13 9

1 1

0

20

40

60

80

100

15-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99

Age

Education

5 2,2 2,2 2,2

19 8,5 8,5 10,7

58 25,9 25,9 36,6

98 43,8 43,8 80,4

44 19,6 19,6 100,0

224 100,0 100,0

3

4

5

6

7

Total

Valid


Cumulative

Percent

Frequency


_______________________________________________________________________________________22

_________________________________________________________________________________________


Table III. Percentiles of the main TOSSA indices in the HEALTHY controls (N=224)

Statistics

224 224 224 207 207 207 207 207 207

0 0 0 17 17 17 17 17 17

91,25 96,01 93,30 97,62 88,73 93,84 92,53 -9,196 -1,272

,5759 ,4174 ,4001 ,2346 ,6952 ,4224 ,5298 ,6295 ,4965

93,40 97,90 95,00 100,0 90,00 95,00 95,00 -7,500 ,000

97,1 100,0 97,5 100,0 95,0 97,5 97,5 ,0 ,0

8,620 6,247 5,989 3,375 10,00 6,078 7,623 9,0568 7,1440

16,3 30,9 72,5 85,0 52,5 72,5 60,0 -43,2 -35,1

100,0 100,0 100,0 100,0 100,0 100,0 100,0 5,3 25,0

76,28 85,70 81,25 90,00 68,50 80,00 77,50 -27,92 -13,16

80,40 89,35 84,40 92,50 72,50 85,00 82,00 -23,08 -8,934

85,80 92,60 88,75 95,00 80,00 90,00 87,50 -15,79 -5,263

88,60 95,05 91,30 97,50 85,00 92,50 90,00 -12,50 -2,748

91,70 96,70 93,70 97,50 87,50 95,00 92,50 -10,26 -2,500

93,40 97,90 95,00 100,0 90,00 95,00 95,00 -7,500 ,000

95,10 99,20 96,30 100,0 95,00 97,50 97,50 -5,000 ,000

96,70 100,0 97,50 100,0 95,00 97,50 97,50 -2,500 ,000

97,10 100,0 98,75 100,0 97,50 100,0 97,50 -2,500 2,632

98,80 100,0 98,80 100,0 100,0 100,0 100,0 ,000 5,657

99,60 100,0 100,0 100,0 100,0 100,0 100,0 2,564 9,556

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles

CS CST DS DST DSS DSB1 DSB2 SADS LADS

Table IV. Percentiles of the main TOSSA indices in the HEALTHY controls (N=224) sequel

Statistics

224 207 207 207 207 207 207

0 17 17 17 17 17 17

98,11 98,57 97,78 98,174 98,133 -,768 -,004

,1552 ,1484 ,1994 ,1676 ,1725 ,2297 ,1940

98,80 99,17 98,33 99,200 99,200 -,800 ,000

99,6 100,0 100,0 99,2 100,0 ,0 ,0

2,323 2,135 2,868 2,4107 2,4825 3,3043 2,7908

81,7 86,7 84,1 83,8 86,8 -12,4 -12,4

100,0 100,0 110,9 100,0 100,0 28,0 13,4

93,30 94,50 92,51 93,667 92,540 -5,085 -5,078

95,40 95,87 94,20 95,867 95,000 -3,333 -1,699

97,10 98,33 95,87 96,667 97,520 -2,049 -,880

97,90 98,33 97,51 98,333 98,333 -1,667 -,800

98,30 99,17 98,33 98,333 98,333 -,814 ,000

98,80 99,17 98,33 99,200 99,200 -,800 ,000

99,20 99,84 99,20 99,200 99,200 ,000 ,000

99,60 100,0 99,20 99,200 99,200 ,000 ,806

99,60 100,0 100,0 100,0 100,000 ,840 ,903

100,0 100,0 100,0 100,0 100,000 1,695 2,573

100,0 100,0 100,0 100,0 100,000 3,114 3,466

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles

RIS RIST RISS RISB1 RISB2 SARIS LARIS


_______________________________________________________________________________________23

_________________________________________________________________________________________


The neurological norm group consisted of 1019 patients with different neurological conditions in which 4 main groups can be found: right hemisphere stroke (N=297), left hemisphere stroke (N=284), traumatic brain injury (N=145) and Other neurological (N=293). This last group consisted of a wide range of disorders like hypoxia/postanoxic encephalopathy (n=65), meningitis, Parkinson’s disease and encephalitis (n=78), brain tumours with or without surgical removal and/or radiation therapy (n=41), brain stem infarction (n=21), mild traumatic brain injury (14), multiple sclerosis (n=41), cerebellar infarction (n=18), and a form of dementia (n=15). Furthermore, 82 patients were tested with the so-called Whiplash Associated Disorders Type II (WAD) of which it is assumed here that there is no proven brain damage. They did have however many complaints about headaches and concentration difficulties and were therefore not considered belonging to the Healthy controls group. The neurological group (including the WAD group) was found in two rehabilitation centres in Leiden and Apeldoorn in the Netherlands. This last centre supplied only patients with traumatic brain injury (since 2001, n=76). To find out if the data collection since 2001 differed from the data collection period before 2001, a statistical analysis was performed on the neurological group (without WAD). The groups before and after 2001 were compared (n=279 vs n=740). No significant differences between these groups could be found on the TOSSA indices nor on the demographic variables sex, education and age. Therefore it can be assumed that there have not been substantial changes in patient groups over a time period of 15 years. Indeed, it would be hard to find any theoretical reasons why such changes would take place. Rehabilitation centres still admit stroke patients and such patients are not very likely to change in their overall problems after a brain injury. In fact, it would also be quite pointless to collect more and more patient data to see if these data would change. There would not be any theoretical reason for such a change.

However, it would be interesting to have more data on the whole range of neurological patient groups and especially to have more variation in age. It is also known that a lot of stroke patients are not admitted to a rehabilitation centre at all (estimates are between 20 and 45%) and it would be interesting to know how the less injured brain injury patients would fare on the TOSSA. One can speculate that including these patients in these norms would lower the incidence of attention problems as found with the TOSSA. Below, graphs and tables will show the demographic data of the 4 large neurological groups (right and left hemisphere stroke, traumatic brain injury and Other neurological patients) and the WAD-group. The distribution of age, sex and education will be displayed.

Right Hemisphere STROKE patients group (N=297):


_______________________________________________________________________________________24

_________________________________________________________________________________________


615

56

80

40

4 0

96

0

20

40

60

80

100

120

20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99

Age

Fre

qu

en

cy

Figure 12. Distribution of age in the right hemisphere STROKE group (N=297); mean: 57.6 yr,

Median: 59.0 yr, range 25-81, SD=11.6 yr

Table V. Distribution of Sex in the Right Hemisphere STROKE group (N=297)

Table VI. Distribution of Education in the RH STROKE group (N=297); mean: 4.8, range 1-7, SD=1.2

Table VII. Percentiles of the main TOSSA indices in the RH STROKE group (N=297)

Education

1 ,3 ,3 ,3

4 1,3 1,3 1,7

42 14,1 14,1 15,8

74 24,9 24,9 40,7

92 31,0 31,0 71,7

67 22,6 22,6 94,3

17 5,7 5,7 100,0

297 100,0 100,0

1

2

3

4

5

6

7

Total

Valid


Cumulative

Percent

Sex: 1 =male; 2 =female

180 60,6 60,6 60,6

117 39,4 39,4 100,0

297 100,0 100,0

1,00

2,00

Total

Valid


Cumulative

Percent


_______________________________________________________________________________________25

_________________________________________________________________________________________


Statistics

297 297 297 297 297 297 297 297 297

0 0 0 0 0 0 0 0 0

60,9 51,8 70,2 -28,9 -9,762 64,3 93,0 63,46 58,40

1,50 1,53 1,55 1,230 1,589 1,44 ,4542 1,459 1,618

64,4 50,4 77,0 -27,5 -6,700 68,8 96,3 68,90 62,50

25,8 26,4 26,6 21,19 27,38 24,9 7,83 25,14 27,88

-,351 ,011 -,681 -,209 -,155 -,481 -1,8 -,426 -,290

,141 ,141 ,141 ,141 ,141 ,141 ,141 ,141 ,141

96,4 100 94,2 137,5 221,9 96,2 43,7 96,2 100,0

3,6 ,0 5,8 -100,0 -100,0 3,8 56,3 3,8 ,0

100,0 100 100,0 37,5 121,9 100 100,0 100,0 100,0

12,9 8,48 16,3 -62,0 -55,2 15,0 76,2 16,68 8,750

23,8 19,2 27,9 -55,1 -42,2 28,8 82,0 28,84 20,46

36,0 25,3 43,7 -45,1 -29,1 42,5 87,3 38,92 30,60

44,8 33,8 56,3 -39,5 -18,0 50,0 92,5 47,10 39,70

52,3 44,0 67,5 -35,7 -12,1 57,5 94,3 57,92 51,34

64,4 50,4 77,0 -27,5 -6,700 68,8 96,3 68,90 62,50

73,3 59,8 85,9 -22,4 -1,620 76,0 97,4 76,28 70,70

79,6 68,8 91,0 -17,5 2,600 81,3 98,1 81,62 78,24

86,1 78,9 96,2 -10,0 7,200 88,8 98,8 86,80 87,50

93,5 89,3 99,2 -4,360 18,02 95,0 99,2 94,20 94,20

96,3 93,5 100 ,000 32,91 97,5 99,6 97,50 97,50

Valid

Missing

N

Mean

Std. Error of Mean

Median

Std. Deviation

Skewness

Std. Error of Skewness

Range

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles

CS CSS CST SACS LACS DS RIS CSB1 CSB2

Table VIII. Percentiles of the main TOSSA indices in the RH STROKE group (N=297) sequel

Statistics

297 102 102 102 102 102 102

0 195 195 195 195 195 195

92,88 93,02 92,34 93,86 91,509 -,622 -2,548

,4628 ,8455 ,8343 ,7242 1,0027 ,4701 ,7526

95,80 96,67 95,00 96,67 95,833 -,848 -1,333

99,2 100,0 99,2 97,5 99,2 ,0 ,0

7,975 8,539 8,426 7,314 10,1264 4,7480 7,6009

56,3 63,3 59,2 55,8 46,9 -15,4 -45,2

100,0 100,0 100,0 100,0 100,0 16,2 24,0

75,00 70,85 74,53 80,00 69,300 -7,415 -12,914

81,20 80,00 79,72 82,96 77,427 -5,992 -9,720

87,10 86,73 88,40 89,33 85,867 -3,453 -5,620

92,02 91,67 91,63 93,33 89,933 -2,522 -3,378

94,20 94,36 93,33 95,87 91,893 -1,695 -1,778

95,80 96,67 95,00 96,67 95,833 -,848 -1,333

97,10 98,33 96,67 97,53 96,667 ,000 -,800

97,90 98,33 97,53 98,33 98,333 ,037 ,000

98,80 99,17 98,33 99,20 99,200 1,894 1,212

99,20 100,0 99,20 99,76 99,200 5,252 3,520

99,60 100,0 100,0 100,0 100,000 9,614 5,043

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles

RIS RIST RISS

RISB

1 RISB2 SARIS LARIS


_______________________________________________________________________________________26

_________________________________________________________________________________________


Left Hemisphere STROKE patients group (N=284):

1420

39

64

2 0

73 72

0

10

20

30

40

50

60

70

80

16-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99

Age

Fre

qu

en

cy

Figure 13. Distribution of age in the Left Hemisphere STROKE group (N=284); mean: 57.2 yr,

median: 59.0 yr, range 16-81, SD=13.8 yr

Table IX. Distribution of Sex in the Left Hemisphere STROKE group (N=284)

Table X. Distribution of Education in the LH STROKE group (N=284); mean: 4.7, range 2-7, SD=1.1


160 56,3 56,3 56,3

124 43,7 43,7 100,0

284 100,0 100,0

1,00

2,00

Total

Valid


Cumulative

Percent

Education

3 1,1 1,1 1,1

31 10,9 10,9 12,0

84 29,6 29,6 41,5

99 34,9 34,9 76,4

53 18,7 18,7 95,1

14 4,9 4,9 100,0

284 100,0 100,0

2

3

4

5

6

7

Total

Valid


Cumulative

Percent


_______________________________________________________________________________________27

_________________________________________________________________________________________


Table XI. Percentiles of the main TOSSA indices in the LH STROKE group (N=284)

Statistics

284 284 87 87 87 87 87 87

0 0 197 197 197 197 197 197

63,87 68,65 81,64 61,32 73,99 68,97 -26,42 -6,325

1,424 1,330 1,990 2,416 2,111 2,311 1,8544 2,362

67,85 73,09 87,50 65,00 77,50 75,00 -25,00 -5,000

93,0 93,8 100,0 80,0 97,5 80,0a -48,5a ,0

23,99 22,42 18,56 22,53 19,69 21,55 17,30 22,03

4,7 6,2 27,5 15,0 27,5 15,0 -69,2 -57,1

100,0 100,0 100,0 97,5 100,0 100,0 19,4 92,9

20,03 26,58 37,00 22,50 35,00 22,50 -52,19 -45,6

26,60 33,80 54,00 27,50 41,50 39,00 -47,47 -39,4

41,90 47,50 65,00 40,00 52,50 47,50 -43,75 -23,4

50,30 57,52 77,50 45,00 70,00 60,00 -37,36 -11,9

60,40 67,44 82,50 53,00 72,50 67,50 -30,56 -7,212

67,85 73,09 87,50 65,00 77,50 75,00 -25,00 -5,000

76,20 80,00 92,50 72,50 82,50 80,00 -20,00 -,526

80,40 85,00 95,00 79,00 87,50 84,00 -15,79 3,290

85,90 89,96 97,50 82,50 92,50 87,50 -12,07 6,857

93,20 93,80 100,0 90,50 97,50 92,50 -5,103 14,29

95,78 97,19 100,0 94,00 97,50 95,00 -2,643 21,79

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles

CS DS DST DSS DSB1 DSB2 SADS LADS

Multiple modes exist. The smallest value is showna.

Table XII. Percentiles of the main TOSSA indices in the LH STROKE group (N=284)

sequel

Statistics

284 284 87 87 87 87 87 87

0 0 197 197 197 197 197 197

63,87 68,65 81,64 61,32 73,99 68,97 -26,42 -6,325

1,424 1,330 1,990 2,416 2,111 2,311 1,8544 2,362

67,85 73,09 87,50 65,00 77,50 75,00 -25,00 -5,000

93,0 93,8 100,0 80,0 97,5 80,0a -48,5a ,0

23,99 22,42 18,56 22,53 19,69 21,55 17,30 22,03

4,7 6,2 27,5 15,0 27,5 15,0 -69,2 -57,1

100,0 100,0 100,0 97,5 100,0 100,0 19,4 92,9

20,03 26,58 37,00 22,50 35,00 22,50 -52,19 -45,6

26,60 33,80 54,00 27,50 41,50 39,00 -47,47 -39,4

41,90 47,50 65,00 40,00 52,50 47,50 -43,75 -23,4

50,30 57,52 77,50 45,00 70,00 60,00 -37,36 -11,9

60,40 67,44 82,50 53,00 72,50 67,50 -30,56 -7,212

67,85 73,09 87,50 65,00 77,50 75,00 -25,00 -5,000

76,20 80,00 92,50 72,50 82,50 80,00 -20,00 -,526

80,40 85,00 95,00 79,00 87,50 84,00 -15,79 3,290

85,90 89,96 97,50 82,50 92,50 87,50 -12,07 6,857

93,20 93,80 100,0 90,50 97,50 92,50 -5,103 14,29

95,78 97,19 100,0 94,00 97,50 95,00 -2,643 21,79

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles




_______________________________________________________________________________________28

_________________________________________________________________________________________


Traumatic BRAIN INJURY patients group (N=145):

55

18

25 25

15

70 0

0

10

20

30

40

50

60

12-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99

Age

Fre

qu

en

cy

Figure 14. Distribution of Age in the Traumatic BRAIN INJURY group (N=145); mean: 39.5 yr,

median: 39.0 yr, range 12-78, SD=17.2 yr

Table XIII. Distribution of Sex in the Traumatic BRAIN INJURY group (N=145)

Table XIV. Distribution of Education in the Traumatic BRAIN INJURY group (N=145); mean: 4.8, range 2-7, SD=1.2


110 75,9 75,9 75,9

35 24,1 24,1 100,0

145 100,0 100,0

1,00

2,00

Total

Valid


Cumulative

Percent


_______________________________________________________________________________________29

_________________________________________________________________________________________


Table XV. Percentiles of the main TOSSA indices in the TBI group (N=145)

Statistics

145 145 50 50 50 50 50 50

0 0 95 95 95 95 95 95

65,23 68,29 80,90 59,15 70,90 69,15 -28,3 -,674

2,045 1,930 2,727 3,333 2,880 3,223 2,920 3,987

73,40 75,00 87,50 62,50 75,00 70,00 -26,2 -3,175

87,6 85,0a 100,0 30,0a 65,0a 55,0a -36,1a -50,0a

24,62 23,24 19,28 23,57 20,37 22,79 20,65 28,20

9,3 10,0 30,0 7,5 25,0 15,0 -81,3 -53,8

99,2 100,0 100,0 97,5 100,0 100,0 12,5 120,0

21,30 22,87 40,00 20,50 31,38 27,75 -65,1 -50,0

28,06 32,50 45,50 30,00 33,00 32,75 -55,4 -33,1

39,70 44,04 62,50 33,00 52,50 48,50 -44,4 -21,4

51,36 56,22 79,00 41,50 65,00 55,75 -41,0 -14,2

60,06 66,77 83,50 55,00 70,00 66,00 -34,5 -6,605

73,40 75,00 87,50 62,50 75,00 70,00 -26,2 -3,175

78,76 80,72 90,00 67,50 79,00 79,00 -20,1 5,129

82,84 84,95 95,00 75,00 82,50 85,00 -15,0 11,66

87,92 88,81 97,50 82,00 89,50 92,50 -9,273 16,56

94,60 95,00 100,0 92,25 97,25 97,25 -2,808 23,91

96,98 97,54 100,0 96,13 100,0 98,63 5,197 40,95

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles



Table XVI. Percentiles of the main TOSSA indices in the TBI group (N=145) sequel

Education

5 3,4 3,4 3,4

13 9,0 9,0 12,4

43 29,7 29,7 42,1

44 30,3 30,3 72,4

30 20,7 20,7 93,1

10 6,9 6,9 100,0

145 100,0 100,0

2

3

4

5

6

7

Total

Valid


Cumulative

Percent


_______________________________________________________________________________________30

_________________________________________________________________________________________


Statistics

145 50 50 50 50 50 50

0 95 95 95 95 95 95

93,87 94,09 92,80 94,238 92,677 -1,295 -1,675

,5915 1,098 1,150 ,9032 1,4581 ,7524 1,2987

97,10 97,93 96,27 96,667 97,533 -,894 -,800

99,6 100,0 99,2 100,0 99,2 -,8 -,8

7,123 7,767 8,130 6,3864 10,3103 5,3206 9,1832

64,6 66,7 62,5 75,0 44,7 -17,7 -47,0

100,0 100,0 100,0 100,0 100,0 17,1 17,0

77,90 77,67 73,20 78,333 68,633 -11,227 -17,37

84,04 80,25 81,85 84,520 82,860 -6,702 -8,686

89,28 87,87 87,50 89,520 86,613 -3,377 -5,147

93,22 91,92 90,82 93,333 91,153 -2,467 -2,622

95,80 95,35 93,68 94,520 94,520 -1,698 -,883

97,10 97,93 96,27 96,667 97,533 -,894 -,800

97,90 99,17 97,50 97,533 98,333 -,800 ,484

98,30 99,20 97,53 98,333 99,200 -,800 1,720

98,80 100,0 98,33 99,200 99,200 ,684 3,264

99,60 100,0 99,20 100,0 100,000 4,281 4,497

99,60 100,0 99,20 100,0 100,000 9,270 12,913

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles


OTHER Neurological patients group (N=293):

3542

65

54

63

33

1 00

10

20

30

40

50

60

70

12-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99

Age

Fre

qu

en

cy

Figure 15. Distribution of Age in the OTHER Neurological patients group (N=293); mean: 50.1 yr,

Range: 12-82, SD=16.0 yr

Table XVII. Distribution of Sex in the OTHER Neurological group (N=293)


_______________________________________________________________________________________31

_________________________________________________________________________________________


Table XVIII. Distribution of Education in the OTHER Neurological group (N=293); mean: 5.0, range 1-7, SD=1.2

Table XIX. Percentiles of the main TOSSA indices in the OTHER neurological group (N=293)

Statistics

293 293 85 85 85 85 85 85

0 0 208 208 208 208 208 208

67,74 71,35 78,06 58,22 70,76 65,47 -25,29 -9,530

1,421 1,346 2,448 2,716 2,260 2,867 3,6095 2,6002

75,60 78,80 87,50 60,00 75,00 72,50 -26,67 -3,333

81,8a 87,5 95,0 85,0 87,5 90,0 -15,8a ,0

24,32 23,05 22,57 25,04 20,84 26,43 33,28 23,97

5,0 5,0 12,5 7,5 17,5 7,5 -85,0 -76,9

100,0 100,0 100,0 100,0 100,0 100,0 220,0 64,3

22,80 27,11 27,50 15,00 32,50 15,75 -69,31 -62,06

28,74 33,75 39,00 22,98 36,50 20,00 -54,61 -41,85

41,40 48,70 60,00 33,00 50,50 38,00 -44,09 -28,86

56,64 62,74 74,00 42,50 62,50 50,00 -37,21 -16,76

66,46 71,29 81,00 53,50 67,50 65,00 -31,52 -7,895

75,60 78,80 87,50 60,00 75,00 72,50 -26,67 -3,333

80,60 83,80 90,00 69,00 82,50 81,50 -19,56 ,000

85,70 87,50 95,00 77,50 85,00 85,00 -15,09 2,874

89,30 91,30 97,50 82,50 87,50 90,00 -11,07 5,882

94,44 96,19 100,0 88,50 95,00 95,00 -4,143 12,434

97,10 98,71 100,0 95,00 97,50 99,25 1,842 19,383

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles




175 59,7 59,7 59,7

118 40,3 40,3 100,0

293 100,0 100,0

1,00

2,00

Total

Valid


Cumulative

Percent

Education

2 ,7 ,7 ,7

5 1,7 1,7 2,4

20 6,8 6,8 9,2

71 24,2 24,3 33,6

97 33,1 33,2 66,8

69 23,5 23,6 90,4

28 9,6 9,6 100,0

292 99,7 100,0

1 ,3

293 100,0

1

2

3

4

5

6

7

Total

Valid

System Missing

Total


Cumulative

Percent


_______________________________________________________________________________________32

_________________________________________________________________________________________


Table XX. Percentiles of the main TOSSA indices in the OTHER neurological group (N=293)

Statistics

293 86 86 86 86 86 86

0 207 207 207 207 207 207

93,56 93,93 92,56 94,126 92,355 -1,329 -1,929

,4182 ,7849 ,7931 ,6559 ,9428 ,5553 ,7050

96,30 97,50 95,43 96,667 96,267 -,865 -,890

99,2 99,2 98,3 97,5 97,5 ,0 ,0

7,158 7,279 7,355 6,0827 8,7428 5,1495 6,5381

49,6 63,3 70,9 78,3 58,9 -24,8 -27,8

100,0 100,0 100,0 100,0 100,0 21,1 16,5

78,53 78,12 76,67 80,280 69,200 -10,32 -14,70

82,50 82,57 80,23 83,333 80,847 -6,937 -9,217

88,80 89,50 86,83 88,333 85,120 -3,932 -4,479

92,90 92,58 90,22 91,780 90,007 -1,871 -2,467

95,00 94,84 93,33 95,867 94,200 -1,669 -1,680

96,30 97,50 95,43 96,667 96,267 -,865 -,890

97,50 98,33 96,67 97,533 97,533 ,000 ,000

98,30 99,17 98,22 98,253 98,253 ,729 ,000

98,80 99,20 98,33 99,200 98,333 1,543 ,878

99,60 100,0 99,20 100,0 99,440 3,008 2,779

99,60 100,0 100,0 100,0 100,0 5,105 8,067

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles


WHIPLASH Associated Disorders (WAD) type II group (N=82):

2825

22

7

0 0 0 00

5

10

15

20

25

30

15-29 30-39 40-49 50-59 60-69 70-79 80-89 90-99

Age

Fre

qu

en

cy

Figure 16. Distribution of Age in the WHIPLASH group (N=82); mean: 34.9 yr, range 18-58, SD=9.7 yr


_______________________________________________________________________________________33

_________________________________________________________________________________________


Table XXI. Distribution of Sex in the WHIPLASH group (N=82)

Table XXII. Distribution of Education in the WHIPLASH group (N=82); mean: 5.2, range 3-7, SD=1.04

Table XXIII. Percentiles of the main TOSSA indices in the WHIPLASH group (N=82)


13 15,9 15,9 15,9

69 84,1 84,1 100,0

82 100,0 100,0

1,00

2,00

Total

Valid


Cumulative

Percent

Education

4 4,9 4,9 4,9

14 17,1 17,1 22,0

32 39,0 39,0 61,0

22 26,8 26,8 87,8

10 12,2 12,2 100,0

82 100,0 100,0

3

4

5

6

7

Total

Valid


Cumulative

Percent


_______________________________________________________________________________________34

_________________________________________________________________________________________


Statistics

82 82 30 30 30 30 30 30

0 0 52 52 52 52 52 52

82,90 84,84 90,33 73,50 84,25 79,58 -18,91 -5,840

1,098 1,061 2,106 2,979 2,094 2,934 2,2874 2,315

83,25 85,04 93,75 73,75 85,00 82,50 -15,19 -5,000

71,9a 95,0a 100,0 52,5a 75,0a 95,0 -10,0 -11,1a

9,946 9,610 11,54 16,32 11,47 16,07 12,53 12,68

50,5 52,5 52,5 50,0 57,5 47,5 -40,0 -36,7

98,8 98,8 100,0 97,5 100,0 100,0 ,0 17,2

63,31 64,47 62,13 50,00 63,00 47,50 -39,71 -34,8

69,94 72,50 75,00 52,50 67,75 55,00 -35,45 -29,4

75,60 77,49 82,50 55,50 73,00 62,00 -33,33 -14,9

77,49 81,23 85,75 60,00 75,00 70,75 -28,53 -10,3

81,40 82,81 90,00 66,00 82,50 77,50 -24,73 -6,286

83,25 85,04 93,75 73,75 85,00 82,50 -15,19 -5,000

85,44 87,26 97,50 81,50 90,00 88,00 -11,67 -1,026

90,24 92,42 100,0 86,75 93,50 94,25 -10,00 2,564

92,76 93,76 100,0 90,00 97,00 95,00 -7,667 3,425

94,60 96,28 100,0 95,00 97,50 95,00 -2,750 7,255

97,50 98,69 100,0 97,50 100,0 100,0 -1,375 14,23

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles



Table XXIV. Percentiles of the main TOSSA indices in the WHIPLASH group (N=82) sequel

Statistics

82 30 30 30 30 30 30

0 52 52 52 52 52 52

97,68 97,54 96,77 97,444 96,860 -,751 -,585

,2342 ,5278 ,4959 ,4465 ,5975 ,4590 ,5268

98,30 98,33 97,53 98,333 98,333 -,807 ,000

98,3a 100,0 98,3 99,2 99,2 ,0 ,0

2,121 2,891 2,716 2,4455 3,2724 2,5142 2,8856

90,4 90,0 90,1 91,7 88,9 -8,4 -7,2

100,0 100,0 100,0 100,0 100,0 4,6 7,2

92,90 90,44 90,49 92,173 89,447 -6,638 -7,014

95,00 91,92 92,73 92,853 90,940 -3,384 -4,942

96,10 95,17 94,36 95,173 93,493 -2,467 -2,710

97,06 96,36 95,26 96,667 96,107 -1,667 -1,438

97,90 97,53 96,67 97,853 97,013 -,884 -,480

98,30 98,33 97,53 98,333 98,333 -,807 ,000

98,80 99,17 98,33 98,853 98,853 ,000 ,000

99,20 100,0 98,33 99,200 99,200 ,000 ,565

99,60 100,0 99,20 99,200 99,200 ,839 ,872

99,60 100,0 100,0 100,00 100,0 2,664 2,450

100,0 100,0 100,0 100,00 100,0 3,711 5,694

Valid

Missing

N

Mean

Std. Error of Mean

Median

Mode

Std. Deviation

Minimum

Maximum

5

10

20

30

40

50

60

70

80

90

95

Percentiles



For an overview of the distribution of the CS-scores (most important TOSSA index) in the 6 large groups (Healthy controls=0, RH Stroke=1, LH Stroke=3, TBI=5, WAD=7, Other=15), see Figure 17. It can be clearly seen that most frequency distributions are not normal. Only in the WHIPLASH group the TOSSA indices CS, DS, DSS, DSB1, DSB2, LADS are normally distributed (Kolmogorov-Smirnov test, p>.05, SPSS Explore). The index SADS (the difference between the quicker and


_______________________________________________________________________________________35

_________________________________________________________________________________________


slower half of the test in the DS index) is normally distributed in the right and left stroke group and in the traumatic brain injury group. It is quite logical the TOSSA distributions are not normal due to the fact that the TOSSA is very simple: see Figure 19 to see the CST distribution within all 6 groups. It is the CS score in the slowest part of the test which shows that the test is quite easy to do.

10

20

30

40

50

Coun

t

0 1 3

5 7 15

25,0 50,0 75,0 100,0

CS

10

20

30

40

50

Coun

t

25,0 50,0 75,0 100,0

CS

25,0 50,0 75,0 100,0

CS

Figure 17. Distribution of the CS index in the 6 largest groups (SPSS interactive graph: histogram)

0

25

50

75

Coun

t

0 1 3

5 7 15

-100,0 0,0 100,0 200,0

SADS

0

25

50

75

Coun

t

-100,0 0,0 100,0 200,0

SADS

-100,0 0,0 100,0 200,0

SADS

Figure 18. Frequency distributions of the SADS index in the 6 largest groups

3.2 Intermezzo about statistics: normal distributions and probability calculations

__________________________________________________________________

Why are frequency distributions so important? In statistical theory it is usually assumed that with most tests the indices are normally distributed. This, of course, is not really the case, especially in the field of neuropsychology. The TOSSA is a test in which we already know from the data that most


_______________________________________________________________________________________36

_________________________________________________________________________________________


healthy people have no problem with doing this test. So the distribution of the main indices is largely skewed. But WHY is normality so important? Well, simply put: with normal distributions you can easily compute/calculate the probability of a test score. Using a standardized Z- or T-score (respectively with a mean of 0 or 50 and a standard deviation of 1 or 10), it can be calculated what the probability of an obtained test score is. For example, a Z-score of 1.96 means that the test score is 1.96 standard deviations away from the mean of 0. This score has a probability of .025 (2.5%) to be part of a normal distribution. The probability that this score falls between Z-score of -1.96 and 1.96 is 1- (2x .025)= .95. Outside this range the probability is 5% or less and that is commonly accepted as a fair cut-off point of what we call ‘abnormality’. So that is also the reason that a score beyond 2 standard deviations of the mean is considered to be highly improbable and therefore it very likely belongs to an ‘abnormal’ distribution (= outside a normal healthy group). This whole reasoning depends largely on the normality assumption and whenever a test score is not normally distributed, the probability calculation is much more difficult.

It would be nice to have other methods to calculate the probability of a test score in neuropsychology. Like logistic regression as is being used a lot in the field of medicine. There only 2 outcomes are important: either a test score is abnormal or normal, so it does belong to only 2 values. Assessing cognitive functions however, seems to be not that easy. In psychology we tend to see a cognitive function as being on a continuum: either more or less present. In medicine this is actually the case as well but…there they have decided more rigorously to label one side of the continuum as ‘abnormal’ and the other side as ‘normal’. By making things so ‘black and white’, medicine has had the advantage of using logistic regression methods to calculate probabilities. Neuropsychological tests cán be set up like this as well: either a neuropsychological test score belongs to a ‘normal healthy population’ or to an ‘abnormal (sick) population’. Of course, thát is the main question for every obtained test score. The most used procedure in (neuro)psychology is to obtain a lot of measurements with a test in groups of healthy and impaired persons. Whenever a test is rather difficult, a very large group of normal (healthy) persons is needed to form a norm group, to try to obtain a normal distribution. It would however, be much simpler to develop a neuropsychological test in which we know beforehand that healthy persons would have no problem with at all and patient groups would indeed have some problems with. The fundamental question is not to establish a normal distribution but to differentiate a healthy distribution from the distribution of the patient groups. And just that can be accomplished by cleverly developing a neuropsychological test. Such a method is much better and more reliable than the statistical way. A second question that is asked in a neuropsychological assessment is how MUCH the obtained ‘abnormal’ score deviates from normal. In other words: how serious is the impairment? Not only does one need to know the deviation from the normal population mean, but also one has to know how the score deviates from the patient groups mean. Just to put a test score into perspective.

3.3 Discriminative power of the TOSSA: sensitivity and specificity

A neuropsychological test has a certain discriminating power: it is its diagnostic value. The TOSSA has to identify as correctly as possible an attention disorder (sensitivity) or not (specificity) in a patient. In the field of medicine, this is usually done by using a so-called ‘gold standard’. A gold standard is a well-used and well-known test of which it is already known that it can identify a certain condition very reliably. Just like a CT-scan that can identify with a very high degree of accuracy that there is a brain tumour. A new test can then be compared to such a gold-standard by using the Receiver Operating Characteristics analysis (ROC-analysis). This analysis can find the highest possible sensitivity and specificity of a new test. This analysis has been done with the TOSSA as well (see later).

Unfortunately, within neuropsychology, such gold standards are rare. For one thing, cognitive functions are measured indirectly and cannot be related directly to any form of tissue damage. Secondly, the overlap with ‘normal’ cognitive processes is quite substantial, making it all the more difficult to differentiate between what is normal and not. To solve this, usually a cut-off score is determined which normally falls below the 5% level of a normal population. Whenever a test-score’s probability is smaller than 5%, it is assumed that it does not fall in a healthy population group. This cut-off score of 5% is in fact 1.96 standard deviations from the mean. In the TOSSA this is the CS-score of 76.3%, which will be considered themost reasonable cut-off point here.


_______________________________________________________________________________________37

_________________________________________________________________________________________


Statistically, there are methods like the ROC-analysis, by which such a cut-off point can be calculated in which the test has the highest possible sensitivity and specificity. Especially, the positive predictive value is important: the chance that a certain condition is present whenever the test result is positive. And the negative predictive value is the opposite: the chance that a specific condition is NOT there whenever the test says it is not (the test is ‘negative’). In the following ROC-analyses it will be investigated whether using a cut-off of CS 76.3% is indeed the most reasonable cut-off point. Three ‘gold standards’ were used for an attention deficit: the WAIS-R Digit Span subtest, the Trail Making test and the criterion of having a brain injury. Digit Span as the gold standard The WAIS-R Digit Span is not only largely used to measure attention but is easy and short to administer. Lezak (2004, p. 351) remarks that the Digit Span Forwards is quite a different task than the Backwards part and so the total score has to be interpreted carefully. That is the reason that the cut-off criterion for this test was set very strict: only the Backwards score was used and it had to be 5 or less to signify an attention disorder. This amounts to be able to say just 2 numbers backwards correctly! Using a Digit Backwards score of 5 or less a state variable Sickdigback=1 could be made. The Digit Span had been administered 443 times together with the TOSSA in the large sample of 1019 patients. The resulting ROC-curve is depicted in Figure 19.

Figure 19. The ROC-curve for 3 different TOSSA-indices in the sample of 443 patients, using Digit Span backwards as ‘gold standard’.

Table XXV. The areas beneath the ROC-curves for the different TOSSA indices, using the Digit Span as the ‘gold standard’ for attention problems


_______________________________________________________________________________________38

_________________________________________________________________________________________


Area Under the Curve

,723 ,024 ,000 ,676 ,770

,717 ,024 ,000 ,669 ,764

,704 ,025 ,000 ,656 ,752

Test Result Variable(s)

CS

DS

RIS

Area Std. Errora

Asymptotic

Sig.b

Lower Bound Upper Bound

Asymptotic 95% Confidence

Interval

The test result variable(s): Concentratiesterkte: RIS *DS, DetectieSterkte, RIS blok1+2 has at least

one tie between the positive actual state group and the negative actual state group. Statistics may

be biased.

Under the nonparametric assumptiona.

Null hypothesis: true area = 0.5b.

It can be clearly seen that most TOSSA indices lie very close together and Table XXV is needed to see what index gives the best values for sensitivity and specificity. That’s the CS index, directly followed by the DS index. Because this CS index represents both the fluctuating speed of the test and the whole test performance, it will be chosen as the most important index of the TOSSA. Unfortunately, too few data were available for the other TOSSA indices to do a reliable ROC-analysis (only DS and RIS had enough data). In the list below a CS of 62.8% represents the highest sensitivity and specificity, namely 77.7% and 60.3% respectively (mean: 69%). This cut-off point of CS seems to be the best value to correctly classify most people in the sample as having an attention deficit (or not). CS sensitivity 1-specificity 61,450 ,782 ,416 61,950 ,782 ,411 62,500 ,782 ,407

62,750 ,777 ,397 62,850 ,773 ,397 62,950 ,769 ,397 63,100 ,760 ,397 63,250 ,760 ,393 63,350 ,755 ,393 63,650 ,751 ,393 64,050 ,747 ,393 64,250 ,742 ,393 64,400 ,738 ,393 To be complete (and to compare) this ROC-analysis was run for the DS index as well. One can see that the DS index has its highest sensitivity (70.7%) and specificity (65.4%) with the value of 71.3% (a mean of 68.08%). DS sensitivity 1-specificity 69,991 ,716 ,374 70,015 ,712 ,355 70,615 ,707 ,355 71,249 ,707 ,350

71,299 ,707 ,346 71,855 ,699 ,346 72,446 ,694 ,346 72,491 ,694 ,341 73,092 ,681 ,336 73,692 ,681 ,332 73,750 ,677 ,327


_______________________________________________________________________________________39

_________________________________________________________________________________________


74,370 ,664 ,327 74,958 ,659 ,327 74,987 ,655 ,327 75,000 ,655 ,327 But…there are doubts in using the Digit Span as a ‘gold standard’ test for an attention deficit. In clinical practice this test is hardly sensitive to any kind of attention deficit. Furthermore, the range of this test (especially the Backwards part) is very small (0-12), leading to statistically lowered correlations with other (attention) tests. And indeed, this ROC-analysis, although statistically sound, does lead to a very strange result. The CS-score of 62.8% (the calculated cut-off point) lies 3.3 standard deviations from the mean of the CS in the healthy norm group (n=224)! This would suggest a very tolerant cut-off point with the risk of not detecting much patients with an attention deficit. But this makes sense: whenever the Digit Span is indeed a not so sensitive test for detecting attention deficits, then the TOSSA will turn out not to be that sensitive as well. Because a 3.3 standard deviations cut-off point is unacceptable a new ROC-analysis was undertaken with the Trail Making Test. Trail Making Test as the gold standard The Trail Making test version A and B seems to be much more sensitive to attention problems than the Digit Span. However, this test does not resemble the TOSSA much because it’s a visual test. Furthermore, a lot of stroke patients do have visuospatial problems and then the TMT becomes rather unreliable to administer. Also the test (just as the Digit Span) cannot be administered to patients with aphasia. Finally, the test length is only a couple of minutes, hardly comparable to the 8 minutes of the TOSSA. But, without having much alternatives, the TMT was administered together with the TOSSA on 100 patients.

In SPSS the variable TMTziekdiff was made on the basis of Sánchez-Cubillo et al (2009) who found that the difference score (TMTB-TMTA) was more sensitive to attention deficits than the much used ratio score (TMTB/TMTA). Again, in clinical practice the ratio score does not do a good job of detecting patients with attention problems. The ROC-analysis was done with the variable TMTziekdiff as the state variable. In Figure 20 the ROC-curve is shown. It is much more ‘blockier’ than Figure 20 but that is obviously due to just having 100 data points instead of 443 in the Digit span-analysis.

Figure 20. The ROC-curve for 3 different TOSSA indices in the sample of 100

neurological patients, using the TMT as the ‘gold standard’


_______________________________________________________________________________________40

_________________________________________________________________________________________


Table XXVI. The areas below the ROC-curves for the different TOSSA indices, using the TMT as the ‘gold standard’ for attention problems


,748 ,049 ,000 ,652 ,844

,751 ,049 ,000 ,655 ,847

,734 ,050 ,000 ,636 ,832


CS

DS

RIS

Area Std. Errora

Asymptotic

Sig.b



Interval



be biased.



Just as in the Digit Span analysis most TOSSA indices lie close together and Table XXVI is needed to see what indices give the best possible results. That’s now the DS index, closely followed by the CS index and then the RIS index. Again the CS index will be chosen as the most important TOSSA index. In the list below it can be seen that a CS score of 64.0% gives the highest sensitivity (90.7% and specificity (54.4%) with a mean of 72.5%. With the Digit span it was respectively 77.7% and 60.3%. However, the cut-off point is still much too low: it’s 3.2 standard deviations of the mean in the healthy sample. CS sensitivity 1-specificity 61,950 ,907 ,526 62,500 ,907 ,509 62,950 ,907 ,491 63,250 ,907 ,474

64,000 ,907 ,456 64,850 ,884 ,456 65,600 ,860 ,456 68,550 ,837 ,456 70,950 ,837 ,439 71,400 ,814 ,439 72,150 ,814 ,421 72,650 ,814 ,404 Using the cut-off point of CS <= 64.0, a 2x2 classification table can be made in which can be seen how sensitivity, specificity, false positives, false negatives and the negative and positive predictive values are calculated. In Excel such a table can be easily made (see Table XXVII), in SPSS this can be done with CrossTabs.

Table XXVII. Classification table for the sample of 100 patients


_______________________________________________________________________________________41

_________________________________________________________________________________________


If the TOSSA is positive, indicating an attention deficit, there is a chance of 88.89% that this is really true (in this sample). If the TOSSA is negative, then there is a chance of 60.94% that this is true. So a chance of 31% that this is NOT true. Which resembles clinical practice in which patients can perform the TOSSA but when they are examined more stringently, a divided attention deficit can still be found. Brain injury as the gold standard Because the cut-off point of the most important TOSSA CS index is far too low when using two commonly used attention tests as the ‘gold standard’, another ROC-analysis has been made with a new criterion using having a brain injury or not. Of course, having a brain injury does not mean that you have most certainly an attention deficit disorder. But it is fairly common after a brain injury. Furthermore, TOSSA’s predictive power for a brain injury can now be studied.

In the total group of 1325 people there are 306 without brain injury (224 healthy controls, 82 WAD type II patients) and 1019 with verified brain injury. In Figure 21 below the ROC-curve can be seen.

Attention deficit using TMT

+ -

true positive false positive all test positives

test + 32 4 36 Positive predictive value:¹ 88,89

false negative

True negative all test negatives

- 25 39 64 Negative predictive value:² 60,94

all disease all healthy everyone

57 43 100 LR+: 6,03508772

pretest odds: 1,3255814

56,1% 90,7% 73,4% posttest odds 8 0,88889

Pre-test Probability: 57,0%


_______________________________________________________________________________________42

_________________________________________________________________________________________


Figure 21. The ROC-curve for the 3 main TOSSA-indices in N=1325 people, using brain injury as the ‘gold standard’

Table XXVIII. The area under the ROC-curve for 3 TOSSA indices in 1325 people


,856 ,012 ,000 ,834 ,879

,865 ,011 ,000 ,844 ,886

,765 ,015 ,000 ,734 ,795


CS

DS

RIS

Area Std. Errora

Asymptotic

Sig.b



Interval



be biased.



In the list below one can see that with a CS score of 81.8% the highest sensitivity (81.4%) and specificity (69.1%) is reached. Strangely enough this cut-off point is only 1.1 standard deviation of the mean of the healthy controls. Still it seems sensitive enough to detect brain injury and it is reasonably specific. Possibly because the TOSSA has a fluctuating speed and most brain injured patients have a slowed information processing. CS sensitivity 1-specificity 81,350 ,817 ,314 81,450 ,817 ,313 81,550 ,814 ,311 81,650 ,814 ,310

81,750 ,814 ,309 81,850 ,804 ,302 81,950 ,804 ,300


_______________________________________________________________________________________43

_________________________________________________________________________________________


82,050 ,804 ,299 82,150 ,801 ,299 82,250 ,794 ,299 82,350 ,791 ,299 With this cut-off point CS <= 81.75 a 2 by 2 classification table can be made to see how sensitivity, specificity, the number of false positives and negatives and the negative and positive predictive values are calculated. In Excel such a table can easily be made (see Table XXII) and in SPSS it can be done with Crosstabs. N.B.: ROC curves are made by taking the cut-off score and higher (only then a test is ‘positive’ for a certain condition). With the TOSSA and other neuropsychological tests it is just the other way around: a score lower than the cut-off score means a positive hit(=1), indicative for a certain impairment. That is why the value under sensitivity in the row of the ROC-analysis is exactly the same as what is presented in the Excel table under specificity (somewhat confusing). But it really is representing sensitivity.

Table XXIX. Classification table for the sample of 1325 people, using brain injury as the gold standard

If the TOSSA is positive, aka: it detects a brain injury (CS <= 81.75%), then there is a probability of 92.2% that there really is a brain injury. If the TOSSA says there is no brain injury, then this probability is less: 44.4%. So when the TOSSA says there is nó brain injury is wise to search further with other tests to see whether there really isn’t any brain injury. In clinical practice, more than once patients can do the TOSSA quite well but have serious problems in doing the TODA. Concentrating on just one thing is indeed quite different from trying to do several things at once, which is what the

TODA is supposed to measure. Summary: Using commonly used attention tests and having a brain injury as gold standards, the above ROC-analyses clearly showed that the cut-off points found in the TOSSA with such analyses aren’t really reasonable and plausible. Until there is no better alternative for a gold standard for an attention deficit, the 5th percentile of the CS index of the TOSSA will remain the most plausible cut-off point of this test. That is the CS score of 76.3% (see Table III, page 21).

Brain injury

+ -

true positive false positive all test positives test + 711 60 771 Positive predictive value: 92,22

false negative true negative all test negatives

- 308 246 554 Negative predictive value: 44,40

all disease all healthy everyone

1019 306 1325 LR+: 3,55848871

pretest odds: 3,33006536

69,8% 80,4% 75,1% posttest odds 11,85 0,92218

specificity sensitivity Pre-test Probability: 76,9%


_______________________________________________________________________________________44

_________________________________________________________________________________________


3.4. Reliability and validity ______________________________________________________

3.4.1. Reliability

Lezak (2004) already states that to determine the reliability and validity of a neuropsychological test isn’t that easy. Test-retest reliability is very difficult to attain when using neuropsychological tests in which usually there are serious test-retest effects due to learning by repetition. Furthermore, a patients group is usually not consistent over time due to spontaneous recovery and/or training effects. On the other hand, it can not be the case that a test at different moments in time suggests quite different conditions. A test score must be reliable and stable over time, unless there are reasons to believe that the brain injury is changing. Fortunately, the TOSSA isn’t a test that is sensitive to any learning effects by repeated exposure. Between 2001 and 2009 data were collected to determine the TOSSA’s reliability. Correlation analyses in a group of 101 people (38 right Stroke, 40 left Stroke, 14 Traumatic Brain Injury, and 9 Other neurological patients, mean age 50.2 yr. sd=13.1, range 16-76 jr, mean educational code 5.0 (1=less than primary school, 7=university, see Appendix II), sd= 1.0, range 3-7, time since injury < 1 year, test-retest-interval between 1 week and 22 months, mean: 4.6 months): good statistically significant test-retest-reliabilities on the indices CS: .84, DS: .82, DST: .75, DSS: .81, DSB1: .77, DSB2: .72, en RIS: .77 (Spearman’s correlation coefficient due to non normally distributed samples). Only the test-retest reliability coefficients of the indices SADS, LADS, SARIS and LARIS were much lower but significant, respectively: .42 (p<=.05), .21 (p<.05), .28 (p<.01) and .21 (p<.05). It is less fortunate that the above test-retest interval is quite long, within a period of 1 year after brain injury in which a lot of spontaneous recovery usually can take place. In a pilot study (Onderwater, 2004) care was taken to shorten this test-retest interval substantially: within 3 weeks a second TOSSA was administered in 27 brain injured patients (the largest part had a stroke, mean age 52.6 yr, average time since injury: 1.5 month (only 2 outliers of 8 months and 4 years not in this average). The test-retest coefficients (Pearson’s R) were very high and significant, p<.01: CS .92, CSS .88, CST .91, RIS .86. However, the SACS was not significant (Pearson’s R: .36) but the LACS was (.49, p<.01) (at that time other test indices were considered to be important). This study does seem to suggest that the TOSSA has high test-retest reliability whenever the spontaneous recovery is controlled for. However, the reliabilities of the indices SACS and LACS are questionable. In 2006 Stutterheim replicated Onderwater’s study in a group of 22 patients with very severe brain injury and very slow information processing. Similar high test-retest reliability scores were found: CS: .92, CST: .89, CSS: .85, RIS: .82, SACS: .30 (ns), LACS: -.20 (ns). Again this suggests that the TOSSA has more than adequate test-retest reliabilities, except for the two indices SACS and LACS. Please note that this test-retest research has largely been done on patient groups. More such research is needed in healthy control groups. Another form of reliability is the split-half reliability,.calculated with the Spearman-Brown coefficient. Performing such analysis on the group of 1325 people yields significant and very high reliability coefficients for the CS index: .94 (comparing the fastest and slowest part) and for the CSB1 versus CSB2 it was .95. For the DS it was .92 (slow vs fast, n=561), and .95 (first-second half). For the RIS it was .92 (slow versus fast) and .88 (first vs second half). The internal consistency, measured with Cronbach’s alpha is .94, both in comparing the slowest and fastest halves and the first and second halves of the test. For the RIS an alpha of .88 is reached when the first and second half of the test are compared; alpha is .92 when comparing the slowest and fastest half of the test (n=562). For the DS (fast-slow) alpha is .91 and .93 (first-second half) (n=561). When all possible comparisons for the CS, DS en RIS indices are taken together in calculating Cronbach’s alpha (comparisons between first and second half and fastest and slowest halves of the test), then alpha reaches .96. Comparing all 3 indices together for the slow-fast comparison and first-second halves comparison yields an alpha of .92 for each comparison. If only the healthy group (n=224) is considered in this kind of analysis the split-half reliabilities are lower but significant (p<.001): CSB1-CSB2 is .82 en CST-CSS: .72. Probably due to a very small range in the healthy group (most values are above 88% considering CS, CSS, CST). For the DS indices the alpha-values are .72 (first-second half) and .50 (fast-slow). For the RIS indices the alpha’s are much lower: .40 (slow vs fast) en .59 (first vs second half).


_______________________________________________________________________________________45

_________________________________________________________________________________________


3.4.2. Validity Validity is comprised of two major concepts:

1. Construct validity: the construct 'concentration’ or ‘attention’ has to be clarified and embedded in a theory about attention within which also relations between this construct and other related constructs has to be laid out. Construct validity answers the question whether operationalisations of the construct do have relations as well, as predicted in the theory. Two derived concepts are convergent validity and divergent validity. Convergent validity examines whether there are correlations between tests which are supposed or assumed to measure the same construct (i.e., attention). Divergent validity investigates the non relationship between tests which are really supposed to measure different constructs (e.g. memory tests measuring memory instead of attention). 2. Criterion validity: In what way are there any correlations between a test score and outcomes on daily life measures, either measured retrospectively, concurrently or in the future. Usually neuropsychologists are most interested in predicting the future (predictive validity) and the concurrent validity (predicting the score on another measure in the present).

The pilot-study of Onderwater (2004) A pilot study of Onderwater (2004) has shed some light on the validity of the TOSSA. Convergent and divergent validity were studied by correlating the TOSSA CS index with other indices of attention tests and non attention tests. Those tests were the WAIS-III Digit Span, the Stroop Color Word test (Hammes, 1971), the Trail Making Test (TMT), the SART (Sustained Attention to Response Task, Manly, Robertson, Galloway, & Hawkins (1999)), and the TODA (Test of Divided Attention, Manual 2009). In Table XXII the correlations of the TOSSA CS index with other attention tests are shown. All correlations (Pearson’s R) were significant except for the SART EC index and the TODA. This last test probably was not administered to enough patients so the correlation could have been non-significant for this reason. The test indices of the Stroop and Trail Making tests were the raw times in seconds, respectively of Color_Word card III and the difference score of the B versus A version of the TMT. As can be seen these correlations lie around .40 and all in the predicted direction. This pilot-study suggests that the TOSSA is indeed measuring an aspect of attention which is measured with other known attention tests as well.


_______________________________________________________________________________________46

_________________________________________________________________________________________


Table XXX. Correlations between the TOSSA CS index and other attention tests

TOSSA CS

Stroop Time 3 Pearson

Correlation

-,407*

Sig. (2-tailed) ,043

N 25

Stroop IS Pearson

Correlation

-,400*


N 25

TMT time B Spearman’s

Rho

-,412*


N 25

TMT IS AB Spearman’s Rho

-,402*


N 25

Digit Span

backwards

Pearson

Correlation

,658**


N 26

Digit Span

backwards score

Spearman’s

Rho

,614**


N 26

Digit Span Forwards +

backwards

Pearson Correlation

,642**


N 26

SART EC Pearson

Correlation

,354


N 15

SART RT Pearson Correlation

-,689**


N 15

TODA TOT Pearson Correlation

,807


N 6


_______________________________________________________________________________________47

_________________________________________________________________________________________


1,00,50,0-0,5-1,0

Component 1

1,0

0,5

0,0

-0,5

-1,0

Co

mp

on

en

t 2 tmbsec

stroop3s

raven

tltscore

tmtbasec

st32sec

dsva

actcs

Component Plot in Rotated Space

A factor-analysis was done later on the data of the Onderwater study (varimax rotation). Figure 24 shows the results.

Figure 22. Principal components factor-analysis with varimax rotation, used on the TOSSA CS

Index (actcs), the Trail Making Test (tmbsec, tmtbasec), the Stroop Color Word test (st32sec, stroop3s), Digit-span (dsva) and other non-attention tests like Tower of London (tltscore) and the Raven Standard Progressive Matrices (raven).

As can be seen very clearly two factors were found of which the 1st factor seems to have the strongest correlations with the Digit Span scores and comes close to the TLT test score and the Raven Total score. That’s a bit strange because the TLT did not correlate with the TOSSA but the Raven did. One explanation is that some concentration is necessary to do the Raven. The 2nd factor loading is related to the Stroop and the Trail Making Test and can be interpreted as the speed factor. However, this factor analysis has been done on only 19 people and further research has to determine whether these 2 factors will show up again with more patients. In any case, the TOSSA has several similarities with the Digit Span. Firstly, both are auditory tasks and full concentration must be paid to both tests. Research into the criterion validity started as well with the pilot-study of Onderwater. She studied the concurrent validity where the criterion was a new attention observation list (the Moss Attention Rating scale, MARS, Whyte, Hart, Bode & Malec, 2003). This MARS consisted of 45 items on a 5-point scale and it was supposed to measure attention components: initiative to action, distractibility and sustained attention (Whyte, personal communication, 2004). A hypothesis was that the TOSSA did measure these attention components as well and that it would correlate with the MARS. In Table XXXIII the correlations of the TOSSA with MARS are presented. Three groups of therapists had scored the MARS: the physical therapists (PT), the occupational therapists (OT) and the nurses (NS). Unfortunately, the number of therapists was small. Still, all the more interesting and promising was that the correlations were quite high: around .44 (only reaching significance with the physical therapists.


_______________________________________________________________________________________48

_________________________________________________________________________________________


Table XXXI. Correlations between MARS (OL) and the TOSSA CS index in 3 groups of therapists: PT (physical therapists), OT (occupational therapists), and NS (nurses)

Tot. OL PT Tot. OL OT Tot. OL NS

TOSSA CS Pearson

Correlation

,445** -,033 ,444

Sig. (2-tailed) ,029 ,879 ,129

N 24 24 13

Two remarks on these results and this study have to be made. First of all, no special training was given before the MARS observation list was filled in by the therapists. Especially, the young less experienced occupational therapists had quite some difficulties with the questions of the MARS. Secondly, the .44 correlation between the TOSSA and the MARS filled in by the nurses suggests that the TOSSA indeed has a significant relationship with the MARS. Especially interesting when the data showed that no other used attention test in this study had any significant correlation with the MARS. Only Digit Span backwards showed a reasonable correlation of .31, p=.146, n.s. within the physical therapists group. In summary: this pilot-study with 27 neurological patients suggests that the concurrent validity of the TOSSA is promising. The correlation between a newly developed Attention rating scale MARS and the TOSSA CS index seems to be relatively high: .44. This is what is often found in other studies using a Continuous Performance Test and attention observation scales (see Riccio et al, 2001, p. 111). The study of Kovács (2009) The largest study, both in duration and number of patients, is the study mentioned in this manual on page 18. In total 1325 people (224 healthy controls, 1019 neurological patients and 82 Whiplash-associated disorder patients) had taken the TOSSA over a time period of more than 14 years. As shown earlier in this manual, other computerized tests were administered together with the TOSSA in the large group of patients. These known tests are: WAIS-R Picture Arrangement, WAIS-R Digit Span, Wisconsin Card Sorting Test, Dutch version of the Rey Auditory Verbal Learning Test and the Tower of London Test. The non-computerized tests are the Trail Making Test and the Stroop Color Word Test. A new computerized test is the Test of Divided Attention (TODA) which resembles TOSSA the most. Some tests were changed a little in their presentation and scoring and these changes will be explained below in detail. Then the correlations between all those tests and the TOSSA will be presented, first the convergent and then the divergent validity. Finally, a factor analysis will be shown on 72 patients. The Stroop Color-Word Test with 100 stimuli was administered with the instruction to read the words column wise. Furthermore, all words had to be read aloud. This test had been taken by 109 neurological patients. The two variables were the time difference between the Color-Word card and the Color card in seconds and the time of the Color-Word card. The Trail Making Test A and B was administered to 100 neurological patients who did not have any signs of visual field defects or a visual inattention. When an error was made the tester immediately gave this feedback so that the patient could restore the error. Meanwhile the timer continued. The two variables analysed were the ratio score TMT B/A and the difference score TMT B – TMT A in seconds (Sánchez-Cubillo et al., 2009). The Dutch version of the Rey Auditory Verbal Learning Test was administered via a computer. All words were clearly spoken aloud and digitalized in a MP3 file to ensure strict standardized presentation. This test was done with 244 neurological patients. The variables analysed were the total number of immediately recalled words (range: 0-75), the number of correctly recalled words after 30 minutes and the number of correctly recognized words. The WAIS-R Picture Arrangement was computerized as well (see Figure 25 for an example) and administered to 338 neurological patients. However, 3 ambiguous items were removed: Flirt, Fish and Taxi. The instructions were exactly the same as in the WAIS-R paper and pencil version but now the patient just had to point at the places where the pictures had to be put. The tester could move the


_______________________________________________________________________________________49

_________________________________________________________________________________________


pictures by using the mouse. This is a subtle difference with the paper and pencil version where it only matters in what sequence the pictures are laid down. In this computerized task, not only the sequence mattered but also the right kind of place and how quickly this was realized. This means that this version of Picture Arrangement becomes much more a planning task in which one has to plan ahead (before just moving the pictures around) to get the right sequence and places right. In this way, this task resembles much more the Tower of London test in which sequencing and planning ahead are important as well. Two variables were calculated in this task. The first is the normal raw score as calculated according to the WAIS-R instructions. The second variable was new and it represented both the number of moves and the number of rightly placed items (sequence). The formula was: Score = (2 * n of pictures on right position – abs[number of moves – correct minimum nr of moves]) / total number of points according to WAIS-R manual. An example: if there is an item with 5 pictures like ENTER and it needs a minimum of 4 moves to get the sequence and all positions right, and the patient has used 5 instead of 4 moves with all 5 positions correct the score is: 2*5 – abs(5-4)/66= 10-1= 9/66= 0.136. Times 100% is 13.6%. The score of 66 is the maximum score when using these 6 items (house, romeo, louie, enter, hunt, hill, robber). The range of this score is 0-100%.

Figure 23. An example of the computerized Picture Arrangement. The WAIS-R Digit Span numbers were digitalized into a MP3 file to ensure strict standardized presentation of the numbers. It was administered to 443 neurological patients. Two variables were used in the analysis: the total raw score (range: 0-24) and the total raw score Backwards (range: 0-12). The Wisconsin Card Sorting Test was digitalized as well and two variables were analysed: the commonly used Perseverative Response (PR) and a new score: the number of times a rule was changed (maximum 6). This was put in this formula: (n of rule changes * 10 / 60)* 100%. When only colour and form were found the score could be: 2*10/60= 0.33*100= 33.3%. This test was administered to 323 neurological patients. The Tower of London Test (TLT) is the computerized version of the original Shallice test (Shallice, 1982, Kovács, 2009). Three blocks are moved on the computer screen by clicking on the left mouse button. This is done by the tester. The patient only has to point at the to be moved blocks and to point out where the blocks have to be placed. This reduces considerably any motor coordination problems patients might have. In this way, this test is a more pure planning test. Furthermore, only one extra attempt to solve an item is permitted (just as in Shallice’s original article). Finally, next to the original and often used score (number of solved items at the first attempt, range: 0-12), a new index has been developed to represent more precisely the planning components of this task. For


_______________________________________________________________________________________50

_________________________________________________________________________________________


each step points are given but more points are given in the first attempt and when the items become more difficult. In this way actual good planning is rewarded much more in points than just using the number of moves or the number of solved items. Data analysis with more than 200 healthy controls shows that this new score follows a normal distribution, in contrast to the much used ‘number of items solved’ score. The index used in the data analysis is the TLTscore: Total score / maximum total score (138) * 100%. The TLT was administered to 486 neurological patients. The Test of Divided Attention (TODA) is a newly developed computerized divided attention test (Kovács, TODA manual, 2009). On a computer screen a sum is displayed vertically (e.g. 2 + 3 = 5) and at the same time a group of 2, 3, or 4 beeps is heard (just as in the TOSSA test). The instruction is that a patient has to judge whether he hears 3 beeps and whether the sum is correct. Only 3 reactions are possible:

1. both are correct (i.e., the sum is correct and there were 3 beeps) 2. one is correct (either the sum or the 3 beeps) 3. both are wrong (i.e., the sum is incorrect and there were 2 or 4 beeps

Reactions are possible via the arrow keys on the numerical keypad. (= both correct, ↓= only one is correct, → = both are wrong) and only the 3 middle fingers of one (dominant) hand have to be used. This task requires quite some divided attention but is much easier than the Paced Auditory Serial Addition Task (PASAT). This TODA has been administered in 255 neurological patients. The d2 test (Brickenkamp & Zillmer, 1998; Bates & Lemay, 2004) is a visual concentration test in which between several distracters a target has to be found and cancelled with a pencil. The target has two forms d” and a d with the 2 accents both under and above. Due to the higher ratio of distracters and a lot of stimuli, this test resembles a continuous performance test. A large difference of course is that the stimuli are static and the speed of the test is self-paced. This test was administered in only 52 neurological patients. The Word Memory Test (Green, Allen, & Astner, 1996) is a simple computerized memory test which has been developed as a symptom validity test. This test was administered in 132 neurological patients.

3.4.2.1. Convergent validity of the TOSSA When it is true that the TOSSA indeed measures some concept like ‘attention’ then other tests supposedly measuring the same construct, should correlate with the TOSSA. As Onderwater (2004) showed in her pilot-study with a small group of patients, there is a reasonable correlation between the TOSSA Index CS and several other attention tests (varying between .41 and .66). In this large study Spearman’s R correlation coefficients were calculated due to non normally distributed variables (however, when there are no extreme outliers and the data are reasonably linearly correlated, Spearman’s R is quite similar to the Pearson’s R coefficient). In Table XXXIV the correlation coefficients can be found between the TOSSA CS index and other attention test indices. Corresponding to Onderwater’s data the highest significant correlations are respectively: the TMT (-.55), the Stroop card 3 (-.51), the TODA (.45), and the Digit Span total and backwards (.42). Remarkable is that the correlation with the Digit Span Forwards, quite a different task than the Digit Span Backwards, has a much lower correlation coefficient (.28). In the pilot-study of Stutterheim (2006) only the correlation between the TMTB and the TOSSA was significant (p<.05): R= -.53. This was probably due to a very small sample of patients: n= 17.


_______________________________________________________________________________________51

_________________________________________________________________________________________


Table XXXII. Correlations between the TOSSA, Trail Making Test, Stroop Color-Word Test, TODA and Digit Span in neurological patients.

Corre lations

1,000 ,967** -,550** -,513** ,449** ,421** ,421** -,368** ,280**

. ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

1325 1325 100 109 273 443 419 109 419

,967** 1,000 -,568** -,516** ,494** ,404** ,407** -,383** ,268**

,000 . ,000 ,000 ,000 ,000 ,000 ,000 ,000

1325 1325 100 109 273 443 419 109 419

-,550** -,568** 1,000 ,711** -,437** -,493** -,519** ,635** -,309**

,000 ,000 . ,000 ,001 ,000 ,000 ,000 ,002

100 100 100 52 51 99 99 52 99

-,513** -,516** ,711** 1,000 -,462** -,431** -,385** ,891** -,231*

,000 ,000 ,000 . ,001 ,000 ,000 ,000 ,019

109 109 52 109 51 103 103 109 103

,449** ,494** -,437** -,462** 1,000 ,288** ,325** -,311* ,145

,000 ,000 ,001 ,001 . ,000 ,000 ,026 ,061

273 273 51 51 273 169 168 51 168

,421** ,404** -,493** -,431** ,288** 1,000 ,828** -,423** ,764**

,000 ,000 ,000 ,000 ,000 . ,000 ,000 ,000

443 443 99 103 169 443 419 103 419

,421** ,407** -,519** -,385** ,325** ,828** 1,000 -,346** ,437**

,000 ,000 ,000 ,000 ,000 ,000 . ,000 ,000

419 419 99 103 168 419 419 103 419

-,368** -,383** ,635** ,891** -,311* -,423** -,346** 1,000 -,264**

,000 ,000 ,000 ,000 ,026 ,000 ,000 . ,007

109 109 52 109 51 103 103 109 103

,280** ,268** -,309** -,231* ,145 ,764** ,437** -,264** 1,000

,000 ,000 ,002 ,019 ,061 ,000 ,000 ,007 .

419 419 99 103 168 419 419 103 419

Correlation Coefficient

Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N

TOSSA CS

TOSSA DS

TMT B

STROOP

card 3 in

sec

TODA total

DIGIT total

DIGIT

backwards

STROOP

card 3-

card 2 in

secondsDIGIT

forwards

Spearman's rho

TOSSA

CS

TOSSA

DS

TMT B

in sec

STROOP

card 3 in

sec

TODA

total

DIGIT

total

DIGIT

backw

STROOP

card 3-

card 2

DIGIT

forw

Correlation is significant at the 0.01 level (2-tailed).**.

Correlation is significant at the 0.05 level (2-tailed).*.

3.4.2.2. Divergent validity of the TOSSA

The construct and concept of ‘attention’ is multi-faceted and probably related to other cognitive functions like memory, problem solving and planning. This makes it more difficult to convincingly show the divergent validity of the TOSSA. However, it should be the case that TOSSA is measuring something different than for example pure memory or pure perception tests. Normally, the procedure in determining divergent validity is that several tests are chosen that represent different constructs like memory or planning. Although it is quite impossible to chose tests which require no attention at all, it is assumed that there are tests in which little attention is required. Such tests have been chosen and the correlations with the TOSSA are shown in Table XXXV. It is obvious that all tests are significantly correlated with the TOSSA CS index, as could be expected. However, when compared to the attention tests these correlations are lower, ranging from .17 to .43. The highest correlation is with the WAIS-R Picture Arrangement (.43). This can be explained in assuming that this computerized format of the test requires a fair amount of concentration and divided attention. The same can be assumed for the Tower of London Test which had a correlation coefficient of .35. As soon as a test requires less attention, the correlation coefficients drop: respectively for the Dutch version of the Rey Auditory Verbal Learning Test (RAVLT) total score, recall score and recognition: .28, .26 and .17. The correlation with the Word Memory Test (WMT) Multipe Choice item is low as well: .23. In contrast, the WMT and the RAVLT correlate highly: .69, as could be expected.


_______________________________________________________________________________________52

_________________________________________________________________________________________


Table XXXIII. Correlation coefficients between the TOSSA (CS), WAIS-R Picture Arrangement, Tower of London test (TLT), Wisconsin Card Sorting Test (WCST), Rey Auditory Verbal Learning Test (15-WORD test) and the Word Memory Test (WMT) in neurological patients.

Corre lations

1,000 ,967** ,432** ,349** ,333** ,279** ,255** ,231** ,166**

. ,000 ,000 ,000 ,000 ,000 ,000 ,008 ,010

1325 1325 338 494 323 244 243 132 243

,967** 1,000 ,444** ,355** ,332** ,273** ,217** ,199* ,150*

,000 . ,000 ,000 ,000 ,000 ,001 ,022 ,019

1325 1325 338 494 323 244 243 132 243

,432** ,444** 1,000 ,433** ,430** ,437** ,411** ,439** ,374**

,000 ,000 . ,000 ,000 ,000 ,000 ,000 ,000

338 338 338 318 261 167 167 120 167

,349** ,355** ,433** 1,000 ,202** ,180** ,217** ,136 ,091

,000 ,000 ,000 . ,000 ,006 ,001 ,126 ,169

494 494 318 494 306 229 228 129 228

,333** ,332** ,430** ,202** 1,000 ,432** ,378** ,338** ,374**

,000 ,000 ,000 ,000 . ,000 ,000 ,000 ,000

323 323 261 306 323 163 162 123 162

,279** ,273** ,437** ,180** ,432** 1,000 ,727** ,694** ,826**

,000 ,000 ,000 ,006 ,000 . ,000 ,000 ,000

244 244 167 229 163 244 243 67 243

,255** ,217** ,411** ,217** ,378** ,727** 1,000 ,560** ,732**

,000 ,001 ,000 ,001 ,000 ,000 . ,000 ,000

243 243 167 228 162 243 243 67 243

,231** ,199* ,439** ,136 ,338** ,694** ,560** 1,000 ,657**

,008 ,022 ,000 ,126 ,000 ,000 ,000 . ,000

132 132 120 129 123 67 67 132 67

,166** ,150* ,374** ,091 ,374** ,826** ,732** ,657** 1,000

,010 ,019 ,000 ,169 ,000 ,000 ,000 ,000 .

243 243 167 228 162 243 243 67 243


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N


Sig. (2-tailed)

N

TOSSA CS

TOSSA DS

PICT ARR

percentage

TLT score

WCST

percentage

15-WT total

15-WT

recall

WMT MC

item

15-WT

recognition

Spearman's rho

TOSSA

CS

TOSSA

DS

PICT ARR

percentage

TLT

score

WCST

percentage

15-WT

total

15-WT

recall

WMT

MC

item

15-WT

recog

Correlation is significant at the 0.01 level (2-tailed).**.

Correlation is significant at the 0.05 level (2-tailed).*.

A factor analysis was performed to see whether there are clusters of tests which can be logically linked to underlying constructs. Figure 24 shows the extracted components. Two components can explain a variance of over 66%. One component can be called ‘concentration’ or ‘holding something in working memory’. Tests like the TOSSA, WAIS-R Digit Span and the Trail Making Test load on this factor. The second component can be called ‘planning’. Here tests like the Tower of London, WAIS-R Picture Arrangement and the Wisconsin Card Sorting Test have higher loadings. This factor-analysis is limited because only 72 patients were included. When using another attention test, the Stroop Color-Word test, the factor structure does not really differ (Figure 25). However, then only 33 patients have done all the tests which limits even more a reliable analysis. In summary: Both the correlation analyses and the factor analyses do show quite convincingly that the TOSSA measures something like ‘attention’, just like other assumed and much used attention tests. Other tests measure other constructs and correlate less with the TOSSA.


_______________________________________________________________________________________53

_________________________________________________________________________________________


Figure 24. Principal components factor analysis with varimax rotation on the TOSSA (cs), Trailmaking B (TMTB), Digit-span (digtotal), Picture Arrangement (plordper), Wisconsin Card Sorting Test (mksttot) and the Tower of London Test (tltscore) in 72 Neurological patients.

Below, the components matrix can be seen in which component 1 probably signifies ‘planning’ and component 2 is ‘attention or working memory’.

Rotated Component Matrix a

,351 ,751

-,144 ,868

,820 ,252

,625 ,250

,799 -,123

-,519 -,649

TOSSA CS

Digit Span total score Picture Arrangement

percentage

Wisconsin CST

percentage

Tower of London test

Trailmaking in sec; part B

1 2

Component

Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization.

Rotation converged in 3 iterations. a.

1,0 0,5 0,0 -0,5 -1,0

Component 1

1,0

0,5

0,0

-0,5

-1,0

TMTB

tltscore

mksttot plordper

digtotal

cs


Component 2


_______________________________________________________________________________________54

_________________________________________________________________________________________


1,00,50,0-0,5-1,0

Component 1

1,0

0,5

0,0

-0,5

-1,0

Co

mp

on

en

t 2

TMTB

tltscore

mksttot

plordper

digtotal

stroop3s

cs


Figure 25. Principal components factor analysis with varimax rotation using the TOSSA (cs), Trailmaking B (TMTB), Stroop Color Word card 3 (stroop3s), Digit-span (digtotal), Picture Arrangement (plordper), Wisconsin Card Sorting Test (mksttot) and the Tower of London Test (tltscore) in 33 neurological patients. Below the components matrix in which component 1 is probably ‘attention or working memory’, component 2 is ‘planning’.

Rotated Component Matrix a

,751 ,276

-,831 -,379

,807 -,273

,247 ,813

,158 ,713

,014 ,757 -,701 -,511

CS STROOP card 3 in sec

Digit Span total score

Picture Arrangement WCST Tower of London Trailmaking in sec; part B

1 2 Component

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Rotation converged in 3 iterations. a.


_______________________________________________________________________________________55

_________________________________________________________________________________________


4. Possible criteria for recognizing reduced effort or malingering

______________________________________________________

Especially with memory and attention tests, it is very difficult to detect if test scores are negatively influenced by insufficient effort or motivation. Below some considerations and help is provided to detect reduced effort with the TOSSA’s parameters. The CS index is the most important TOSSA index which tells you if there really is an attention deficit and how serious this deficit is. This severity of a deficit is determined by looking at the frequency distribution of the CS index within the group of healthy controls (n=224). Because this group is still relatively small, deciles are used to categorize the severity. A CS score on or under the 5th percentile is seen as a deficit. How severe this deficit is, is compared to the right stroke patients group (n=297) because this group usually has the most serious attention disorders. The attentional network of Posner and Peterson’s model (1990) is situated mainly in the right hemisphere. Below you’ll find the used interpretation of the CS score: CSscore 0 to 24.4%: very serious concentration deficit CSscore >=24.4 to 44.5%: serious concentration deficit CSscore >=44.5 to 52.3%: moderate concentration deficit CSscore >=52.3 to 76.3%: slight concentration deficit _______________________________________________________________deficit___________ CSscore >=76.3 to 80.4%: obvious concentration problem 1st decile CSscore >=80.4 to 85.8%: quite obvious concentration problem: 2nd decile CSscore >=85.8 to 88.6%: moderate concentration problem: 3rd decile CSscore >=88.6 to 91.7%: slight concentration problem: 4th decile _______________________________________________________________problem_________ CSscore >=91.7 to 93.4%: average concentration: 5th decile CSscore >=93.4 to 95.1%: sufficient concentration: 6th decile CSscore >=95.1 to 96.7%: more than sufficient concentration: 7th decile CSscore >=96.7 to 97.1%: good concentration: 8th decile CSscore >=97.1 to 98.8%: very good concentration: 9th decile CSscore >=98.8 t/m 100%: excellent concentration: 10th decile N.B.: A note of caution: neuropsychologically it is not wise to conclude neurological and/or cognitive damage solely on the basis of just one test. More tests are needed, just as observations, medical history, and neurological data. However, if the TOSSA says that there is an attention deficit, this is almost certainly true (88.9% positive predictive value using TMT as gold standard, see Table XXVII). The chances of a brain injury are then very high: 92.9% (Table XXIX). When the TOSSA concludes that there is NO attention deficit, there is a 61% probability that this is true (Table XXVII) and there is a 44.4% probability that there really is no brain damage (Table XXIX). So when the test is negative, further research with more (difficult) attention tests like the TODA is advisable. Although the CS index alone can signal serious attention problems, it cannot identify whether there are any external factors involved in a lowered CS score. These can be negative emotions, poor motivation or effort or even deliberate attempts to lower the CS score. Some things can be said, though:

1. It is rarely the case that patients press on the distracter 2. Whenever there are at least as much reactions to 2 beeps as to 4 beeps, one must interpret the CS index with caution because this suggests malingering.

2. the most common errors are wrong reactions on 4 beeps and omissions in the fastest part of the test.

3. In the graphical display (Figure 6, page 9) the line almost always is gradually descending from left to right. Especially the indices RIS and DST are very rarely lower than CS and DSS.

To help with detecting malingering or a reduced effort on the TOSSA the following two Tables might be useful:


_______________________________________________________________________________________56

_________________________________________________________________________________________


Table XXXIV. Prevalence of the TOSSA-indices in neurological patients

Statistics

1019 325 325 325 325 324 324

0 694 694 694 694 695 695

92,724 93,22 91,93 -1,225 -2,133 ,727438 ,930004

95,800 96,67 95,00 -,914 -,874 ,729730 ,962963

7,8799 8,384 8,190 5,3391 7,5356 ,237920 ,241069

49,6 57,5 59,2 -24,8 -47,0 ,0000 ,0000

100,0 100,0 100,0 27,3 30,6 3,2000 2,2000

75,00 72,50 73,3 -11,36 -15,6 ,370370 ,500000

80,400 80,00 79,60 -6,636 -9,264 ,481935 ,615385

86,700 86,73 85,83 -4,160 -5,323 ,558824 ,743590

91,700 91,67 90,80 -2,571 -2,643 ,605662 ,857143

94,200 95,00 93,31 -1,718 -1,709 ,666667 ,925000

95,800 96,67 95,00 -,914 -,874 ,729730 ,962963

97,100 98,33 95,87 -,800 ,000 ,789474 1,00000

97,900 99,17 97,53 ,000 ,000 ,842105 1,03280

98,800 99,20 98,33 1,219 1,709 ,891892 1,09091

99,200 100,0 99,20 3,663 4,060 ,950000 1,16065

99,60 100,0 100 9,528 7,416 1,00000 1,31579

Valid

Missing

N

Mean

Median

Std. Deviation

Minimum

Maximum

4

10

20

30

40

50

60

70

80

90

96

Percentiles

RIS RIST RISS SARIS LARIS

DSS/

DST

DSB2/

DSB1

Table XXXV. Prevalence of the TOSSA-indices in the healthy controls

Statistics

224 207 207 207 207 207 207

0 17 17 17 17 17 17

98,11 98,57 97,78 -,768 -,004 ,908041 ,987278

98,80 99,17 98,33 -,800 ,000 ,925000 1,00000

2,323 2,135 2,868 3,3043 2,7908 ,090568 ,071440

81,7 86,7 84,1 -12,4 -12,4 ,5676 ,6486

100,0 100,0 110,9 28,0 13,4 1,0526 1,2500

92,90 94,17 91,19 -5,577 -5,142 ,712901 ,848187

95,40 95,87 94,20 -3,333 -1,699 ,769231 ,910662

97,10 98,33 95,87 -2,049 -,880 ,842105 ,947368

97,90 98,33 97,51 -1,667 -,800 ,875000 ,972523

98,30 99,17 98,33 -,814 ,000 ,897436 ,975000

98,80 99,17 98,33 -,800 ,000 ,925000 1,00000

99,20 99,84 99,20 ,000 ,000 ,950000 1,00000

99,60 100,0 99,20 ,000 ,806 ,975000 1,00000

99,60 100,0 100,0 ,840 ,903 ,975000 1,02632

100,0 100,0 100,0 1,695 2,573 1,00000 1,05657

100,0 100,0 100,0 3,448 3,544 1,0256 1,10652

Valid

Missing

N

Mean

Median

Std. Deviation

Minimum

Maximum

4

10

20

30

40

50

60

70

80

90

96

Percentiles

RIS RIST RISS SARIS LARIS

DSS/

DST

DSB2/

DSB1

When the likelihood exceeds the 4% boundary it is assumed that a score is highly improbable in the current population. Careful consideration and validity testing is then highly advisable.


_______________________________________________________________________________________57

_________________________________________________________________________________________


5. TOSSA versus other attention tests ______________________________________________________

The TOSSA as a continuous performance test is clearly different from other much used attention tests. One of the largest differences is that the TOSSA is dynamic: the demands on a patient are fluctuating over time and the stimuli are short-lived (volatile). Secondly, the TOSSA takes at least twice as long as most other attention tests like the Trail Making Test, the Stroop Color Word Test and the WAIS-R Digit Span. Finally, the TOSSA has auditory stimuli instead of visual stimuli which are more common. To get an idea of the clinical usefulness of the TOSSA to detect concentration disorders, in this large sample of 1325 patients and healthy controls, several other (much used) attention tests were compared with the TOSSA. They were the WAIS-R Digit Span, the Trail Making Test A and B, the Stroop Colour-Word Test and the d2 test (Brickenkamp & Zillmer, 1998). Only 99 patients had been administered the TOSSA, TMT and the Digit Span. These numbers are smaller because only in 2008 data collection for the TMT, Stroop and d2 test had begun. Tables XXXIII till XL show the detection rate of the tests.

For the TOSSA the cut-off score of 76.3% was used (see page 40). For the Trail Making Test the difference score TMTB-TMTA had to be 60 seconds or more to conclude that there is an attention deficit (see Sánchez-Cubillo et al., 2009), the TMTratio (TMTB/TMTA) had to be 3 or higher and the percentile score had to be 10 or lower (determined according to Dutch norms, Schmand et al., 2012). For the Stroop Color Word test the criterion for an attention deficit was the difference between Card 3 and Card 2 which had to be the 10th percentile or lower compared to a Dutch norm group (Schmand et al.,2012). For the WAIS-R Digit Span the total score had to be 8 or lower to conclude an attention deficit and Digit Span Backwards score had to be 5 or lower.

In the next Tables XXXVI to XXXIX one can see how the tests succeed in detecting the attention deficit in patients.

Table XXXVI. Detection rate of the TOSSA compared to 2 much used attention tests (Trail Making Test en WAIS-R Digit Span in 99 patients

Table XXXVII. Detection rate of the TOSSA compared to 3 much used attention tests (Trail Making Test, WAIS-R Digit Span and the Stroop Colour-Word Test in 52 patients

TOSSA TMT diff Digit Span backwards

TMT perc

TMT ratio

Digit total

Stroop perc

Number of attention deficit patients in which all tests were administered

52 52 52 52 52 52 52

Number of patients detected as having an attention deficit

26 26 22 18 15 7 6

Detection percentage 50,0% 50,0% 42,3% 34,6% 28,8% 13,5% 11,5%

TMT Diff TOSSA Digit Back TMT perc

TMT ratio

Digit Total


99 99 99 99 99 99


56 47 43 41 35 16

Detection percentage 56,6% 47,5% 43,4% 41,4% 35,4% 16,2%


_______________________________________________________________________________________58

_________________________________________________________________________________________


To compare even better, separate tables for each test were setup because the TOSSA and the Digit Span had been administered in 443 patients but only in 100 patients the TOSSA ánd TMT had been administered together (see Tables XXXVIII and XXXIX).

Table XXXVIII. Detection rate of the TOSSA, compared to the Trail Making Test and the WAIS-R Digit Span separately, in 100 neurological patients

TMT Diff TOSSA TMT perc

TMT ratio

Digit Back Digit Total

TOSSA


100 100 100 100 443 443 443


57 47 42 36 214 66 255

Detection percentage 57,0% 47,0% 42,0% 36,0% 48,3% 14,9% 57,6%

Table XXXIX. Detection rate of the TOSSA compared to the d2 test and the WAIS-R Digit Span in 49 neurological patients

TOSSA Digit Span backwards

d2 test Digit total


49 49 49 49


26 22 10 7

Detection percentage 53,1% 44,9% 20,4% 14,3%

These tables show that the TOSSA seems to be one of the most sensitive tests to detect an attention deficit. More specifically, when the normally used indices of the Digit Span (total score) and of the Trail Making Test (ratio score or percentile score) are used to detect an attention deficit, the TOSSA clearly IS the most sensitive test. These data corroborate findings in clinical practice in which most used attention tests fail to find attention problems although stories of patients themselves and their relatives do reveal such problems. TOSSA’s differentiating power In a Master’s thesis by Brouwer and Geuke (2006), the TOSSA was capable to differentiate between patients with different types of sleep disorders. Both the TOSSA index CS and RIS were capable of this. The TOSSA was compared to the Digit Span, the WAIS Digit-Symbol subtest, an experimental visual vigilance task which lasted 10 minutes and a choice-reaction time task of 10 minutes. Although the vigilance and reaction time task could differentiate as well between groups of sleep disorders, this was much more difficult and with just one of many used test parameters. Digit Span and the WAIS Symbol-Digit had no discriminating power in this study. In a study by Kouijzer, de Moor, Gerrits, Congedo, and van Schie (2009) the TOSSA was able to detect significant improvements in children with autism after being trained with neurofeedback, in contrast to no improvement in concentration in a randomized control group.


_______________________________________________________________________________________59

_________________________________________________________________________________________


6. Literature

________________________________________________________

Baddeley, A.D. (1986). Working Memory. Oxford: Clarendon Press. Bates, M.E., & Lemay, E.P. (2004). The d2 test of attention: Construct validity and

extensions in scoring techniques. Journal of the International Neuropsychological Society, 10, 392-400.

Braver, T.S. Working memory. In Smith, E.E. and Kosslyn, S.M. (Eds.) Cognition: Mind and Brain. New York: Prentice Hall, 2007. pgs. 239-278. Braver, T.S., & Barch, D.M. (2002). A theory of cognitive control, aging cognition, and neuromodulation. Neuroscience and Biobehavioral Reviews, 26, 809-817. Brickenkamp, R. & Zillmer, E. (1998). The d2 Test of Attention. First US edition.

Seattle: Hogrefe & Huber Publishers. Brouwer, A., & Geuke, M. (2006). The diagnostic value of neuropsychological tests in sleep disorders. Doctoral thesis, University of Amsterdam Green, P., Allen, L.M., & Astner, K. (1996). The Word Memory Test: A user’s

guide to the oral and computer-administered forms. Durham: CogniSyst, Inc. Greenberg, L.M. (1996-1999). The Test of Variables of Attention-Auditory (TOVA-A). Los Alamitos, CA: Universal Attention Disorders. Hammes (1971). Dutch adaptation of the Stroop Color-Word test. In Bouma, A., Mulder, J., Lindeboom, J. (1996) Neuropsychologische diagnostiek. Handboek. Lisse: Swets en Zeitlinger. Kinsella, G.J. (1998). Assessment of attention following traumatic brain injury: a review. Neuropsychological Rehabilitation, 8(3), 351-375. Kouijzer, M.E.J., de Moor, J.M.H., Gerrits, B.J.L., Congedo, M., & van Schie, H.T. (2009). Neurofeedback improves executive functioning in children with autism spectrum disorders. Research in Autism Spectrum Disorders, 3, 145-162. Kovács, F. (2009). Tower of London Test, Manual, Pyramid Productions. Kovács, F. (2009). Test of Divided Attention. Manual, Pyramid Productions.

Lezak, M.D. (2004). Neuropsychological assessment (3rd edition). New York: Oxford University Press. Manly, T., Robertson, I.H., Galloway, M., & Hawkins, K. (1999). The absent mind: further investigations of sustained attention to response. Neuropsychologia, 37, 661-670. Miller, E.K., & Cohen, J.D. (2001). An integrative theory of prefrontal cortex functions. Annual Review of Neuroscience, 24, 167-202. Norman, D.A., & Shallice, T. (1986). Attention to action: Willed and automatic control of behavior. In T. Shallice and D.A. Norman (Eds.), Consciousness and self-regulation, p. 1-18. New York: Plenum Press. Onderwater, A. (2004). Validating a new attention test: the TOSSA; a pilot-study. MA Thesis University of Leiden. Posner, M.I., & Peterson, S.E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 1, 25-42.


_______________________________________________________________________________________60

_________________________________________________________________________________________


Riccio, C.A., Reynolds, C.R., & Lowe, P.A. (2001). Clinical applications of continuous performance tests. Measuring attention and impulsive responding in children and adults. New York: John

Wiley & Sons, Inc. Riccio, C.A., Reynolds, C.R., Lowe, P., & Moore, J.J. (2002). The continuous performance test: a

window on the neural substrates for attention?. Archives of Clinical Neuropsychology, 17, 235-272.

Rosvold, H.E., Mirsky, A., Sarason, I., Bransome, E.D., Jr., & Beck, L.H. (1956). A

continuous performance test of brain damage. Journal of Consulting Psychology, 20, 343-350.

Sánchez-Cubillo, I., Periáñez, J.A., Adrover-Roig, D., Rodríguez-Sánchez, J.M., Ríos-Lago, M.,

Tirapu, J., & Barceló, F. (2009). Construct validity of the Trail Making Test : Role of task-switching, working memory, inhibition/interference control, and visuomotor abilities. Journal of the International Neuropsychological Society, 15, 438-450

Schmand B, Houx P & de Koning I. Normen voor Stroop Kleurwoord Tests, Trail Making Test en

Story Recall van de Rivermead Behavioral Memory Test. Sectie Neuropsychologie, Nederlands Centrum voor Psychologen, Amsterdam, Ref Type: Report, februari 2012

Shallice, T. (1982). Specific impairments of planning. Philosophical Transactions of the Royal Society of London, 298, 199-209. Shallice, T. (1988). From neuropsychology to mental structure. New York: Cambridge Press. Shiffrin, R.M., & Schneider, W. (1977). Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory. Psychological Review, 84, 127-188. Stutterheim, E. (2006). Aandacht testen – tot op de bodem. Crossvalidatie van verschillende aandachtstests bij een populatie van chronische NAH-patiënten met ernstige meervoudige beperkingen. Doctoraalscriptie, Universiteit van Leiden. Verhage, F. (1964). Intelligentie en leeftijd: Onderzoek bij Nederlanders van twaalf tot zevenenzeventig jaar. Proefschrift [Intelligence and age: Research with Dutch people aged 12 to 77. Doctoral dissertation]. Assen: Van Gorcum. Whyte, J., Hart, T., Bode, R.K., Malec, J.F. (2003) The Moss attention rating scale (MARS) for traumatic brain injury: Initial psychometric assessment. Archives of Physical Medical Rehabilitation, 84, 268-276.


_______________________________________________________________________________________61

_________________________________________________________________________________________


Appendix I: coding system for education and diagnosis ______________________________________________________

In the New Patient screen (most left icon in the menu (Fig. 2, page 6)) a specific coding system has been used in the norming study. This helps in further collecting norm data. However, sometimes a diagnosis is difficult to sort into the 18 used numbers and can then be written down in plain text. N.B.: The format in the Birth day input screen adjusts itself to the country in which it is used. In Europe it is DD-MM-YYYY, in the United States it should be MM-DD-YYYY, so please use 01-03-1954 and not 1-3-54. With the Sex input screen only M (Male) or F (female) is allowed (small or capitols). Education according to the Verhage system (1964) 1. less than primary school/ primary school not completed; less than 8 years of education 2. primary school completed; 8 years of education 3. primary school completed but not completed further education; between 8 to 10 years of

education 4. education at a level lower than lower general secondary education (MAVO), e.g. lower economic

and administrative education (LEAO), domestic science school (LHNO), technical school (LTS); between 10 to 12 years of education

5. lower general secondary education (MAVO), intermediate technical school (MTS), intermediate business education (MEAO); between 12 to 14 years of education

6. higher general secondary education (HAVO), pre-university eduction (VWO), higher vocational education (HBO) with certificate; between 13 and 17 years of education

7. University degree with certificate Coding for diagnosis 1 Right hemisphere STROKE 3 Left hemisphere STROKE 5 Traumatic Brain Injury (abnormal or normal brain scan but duration of impaired consciousness more than 15 minutes) 6 mild Traumatic Brain Injury (normal brain scan with less than 15 minutes of impaired

consciousness 7 Whiplash Associated Disorder (WAD) type II 8 Multiple Sclerose (MS) (all types: relapsing/remitting, primary or secundary progressive) 9 Systemic Lupus Erythematosus (SLE) 10 Brain stem stroke (basal ganglia, pons, thalamus) 11 Cerebellum stroke, either left or right 12 Tumor-/cyst- extirpation or -radiation 13 Hypoxic encephalopathy (e.g., after cardiac arrest and resuscitation) 14 Diffuse general cognitive damage / forms of general encephalopathy and dementia 15 Other diagnoses not to be placed in this categorization system 16 Parkinson’s disease or Parkinsonism 17 Meningitis 18 Encephalitis

t.o.s.s.a test of sustained selective...

Documents