supp.apa.org · web viewa parallel analysis yielded a five-factor solution. an initial efa...

Cognitive Dimensions in Children with Problems in Attention, Learning and Memory

1

Supplementary Materials

Cognitive Dimensions for the Whole Sample, All Ages

Supplementary Table 1

Correlations between Cognitive and Learning Measures for the Whole Sample, All Ages

Variable 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1. Alliteration 2. Rapid Naming .29** 3. Nonword Repetition .38** .12** 4 .Digit Recall .37** .14** .54** 5. Dot Matrix .22** .19** .17** .31** 6. Backward Digit .41** .27** .32** .47** .40** 7. Mr X .20** .11** .19** .26** .37** .35** 8. Following Instructions .27** .26** .25** .24** .26** .28** .24**

9. Delayed Recall .34** .14** .30** .30** .19** .27** .25** .25** 10. SRT .27** .19** .17** .08* .20** .21** .15** .20** .13** 11. Vigil .29** .21** .17** .14** .19** .20** .21** .22** .18** .31** 12. Cancellation .24** .32** .15** .15** .25** .27** .20** .19** .19** .21** .23** 13. Matrix Reasoning .29** .08* .27** .29** .35** .33** .32** .30** .28** .18** .21** .23** 14. Reading .48** .29** .49** .44** .28** .42** .16** .25** .30** .18** .17** .22** .32** 15. Spelling .37** .26** .40** .37** .23** .40** .18** .19** .24** .12** .13** .20** .27** .79** 16. Maths .37** .22** .38** .39** .36** .44** .36** .29** .24** .22** .28** .32** .52** .51** .48**

Note. * p < .05. ** p < .01 uncorrected

2

Exploratory Factor Analysis (EFA)


Three- vs Four-Factor Models for the Whole Sample, All Ages

Four-Factor Model Three-Factor Model

Factor Loadings

F1/ Phonological Processing

F2/ Executive Functions

F3/Processing

Speed

F4/ Attention

Processing

F1/ Phonological Processing

F2/Processing

Speed

F3/Executive Functions

Alliteration 0.31 0.01 0.13 0.41 0.3 0.51 -0.04Rapid Naming 0 -0.01 0.92 0 -0.04 0.45 0.06Nonword Repetition 0.63 -0.11 -0.02 0.23 0.65 0.16 -0.11Digit Recall 0.83 0.09 0.01 -0.1 0.79 -0.09 0.12Dot Matrix 0 0.69 0.03 -0.04 -0.01 0 0.69Backward Digit Recall 0.31 0.39 0.12 0.06 0.3 0.17 0.37Mr X 0.03 0.53 -0.05 0.08 0.05 0.04 0.52Following Instructions 0.09 0.22 0.13 0.23 0.09 0.3 0.21Delayed Recall 0.24 0.12 0 0.26 0.26 0.24 0.09Simple Reaction Time -0.11 0.13 0.06 0.46 -0.12 0.5 0.08Vigil -0.05 0.13 0.08 0.44 -0.06 0.48 0.09Cancellation -0.04 0.25 0.23 0.19 -0.07 0.37 0.22Matrix Reasoning 0.09 0.42 -0.11 0.24 0.13 0.14 0.39

3

Links between Cognition and Learning for the Whole Sample, All Ages


Correlations between Cognitive Factor Scores and Learning Outcomes for the Whole Sample, All Ages

Note. * p < .05. ** p < .01.

4

Variable 1 2 3 4 5

1. Phonological Processing

2. Speed .81**

3. Executive Function .89** .90**

4. Reading .57** .54** .54**

5. Spelling .48** .45** .47** .79**

6. Maths .56** .58** .62** .51** .48**

Supplementary Figure 1

Relationships between Cognitive Factors and Scores on Learning Measures for the Whole Sample, All Ages

Note. Distributions are displayed on the diagonal, Pearson correlations in the upper triangle and simple linear regressions in the lower triangle.

5


Path Models Predicting Learning from Cognitive Factors for the Whole Sample, All Ages

Note. *

p

< .05.

** p

< .01.

Children with and without ADHD in the Whole Sample, All Ages


Fit Statistics for Measurement Invariance in Children With and Without ADHD, All Ages

Model n χ2 df p χ2 p CFI RMSEA (90% CI) SRMR AIC

6

Path Estimate SE z p Fully Standardized EstimateReading

Phonological Processing 0.783 0.115 6.827 <.001** 0.445Speed 0.943 0.214 4.406 <.001** 0.304Executive Functions -0.294 0.201 -1.461 .144 -0.133R2 0.348

Spelling

Phonological Processing 0.449 0.108 4.138 <.001** 0.307Speed 0.335 0.180 1.86 .063 0.13Executive Functions 0.142 0.182 0.78 .436 0.077R2 0.242

Maths

Phonological Processing 0.010 0.100 0.099 .921 0.006Speed 0.304 0.202 1.505 .132 0.101Executive Functions 1.136 0.193 5.889 <.001** 0.528R2 0.392

df

ADHD 255 155.59 - - - - - - - - -

No ADHD 550 165.47 - - - - - - - - -

Configural Invariance 805 321.06 124 <.001 - - - 0.8950.063

[0.054 0.071]0.054 67207

Metric Invariance 805 329.49 134 <.001 7.33 10 .694 0.8960.060

[0.052 0.068]0.057 67195


Descriptive Statistics for Children with ADHD and Possible ADHD, All Ages

Children with ADHD (N=198) Children with Possible ADHD (N=57) Group ComparisonsMeasurement n min max M SD SE n min max M SD SE t p d

7

Phonological Processing Alliteration 194 70 103 92.2 9.57 0.69 55 70 101 92.18 9.83 1.32 0.01 .989 0.002 Rapid Naming 191 0 131 90.98 16.16 1.17 54 69 121 87.35 14.41 1.96 1.49 .138 0.231 Non-word Repetition 169 45 123 82.78 21.84 1.68 49 45 122 86.88 21.4 3.06 -1.16 .246 -0.19Processing Speed Simple Reaction Time 180 1 19 8.09 4.35 0.32 50 1 16 7 3.43 0.49 1.63 .104 0.262WM/STM Digit Recall 197 60 149 91.03 15.48 1.1 57 64 128 94.09 15.65 2.07 -1.31 .192 -0.198 Dot Matrix 197 56 135 89.83 15.84 1.13 57 62 125 87.91 14.44 1.91 0.82 .411 0.124 Backward Digit Recall 195 64 137 90.79 12.57 0.9 55 64 123 93.32 13.17 1.78 -1.31 .193 -0.2 Mr X 196 62 134 95.74 14.77 1.06 56 61 134 99.6 15.25 2.04 -1.71 .088 -0.26 Following Instructions 189 -9.11 18.58 0.3 3.99 0.29 54 -7.76 7.74 0.24 3.24 0.44 0.09 .928 0.014Episodic Memory Delayed Recall 188 1 17 7.51 3.23 0.24 55 2 16 8.16 3.11 0.42 -1.33 .185 -0.204Executive Function Vigil 181 3 16 8.03 3.31 0.25 50 4 16 8.62 3.66 0.52 -1.08 .280 -0.174 Cancellation 194 1 19 10.42 3.55 0.25 54 2 19 11.35 3.26 0.44 -1.73 .085 -0.267Nonverbal Reasoning Matrix Reasoning 198 23 72 43.32 9.98 0.71 57 25 73 47.19 10.49 1.39 -2.55 .011* -0.385Leaning Measures Reading 193 49 129 88.06 17.25 1.24 53 61 134 93.62 15.49 2.13 -2.12 .035* -0.331 Spelling 192 48 131 85.06 14.89 1.07 56 50 119 88.7 14.49 1.94 -1.62 .107 -0.247 Maths 197 42 156 84.78 17.93 1.28 55 55 136 92.45 19.14 2.58 -2.76 .006** -0.423

Note. Residual scores were calculated for the following instructions task. WM= Working Memory; STM = Short-term Memory.

* p < .05. ** p < .01.


Distribution of Cognitive and Learning Scores for Children with ADHD, Possible ADHD, and No ADHD, All Ages

8

Note. Scores are scaled with a mean = 0 for simplicity. dx_ADHD = children with ADHD diagnosis, no_ADHD = children with no ADHD diagnosis, poss-

ADHD = children who were awaiting final diagnostic consultation and therefore possibly had ADHD. Variable names are: RAN = Rapid Automatic Naming,

CNRep = Nonword Repetition, SRT = Simple Reaction Time.


Fit statistics for Measurement Invariance in Children With and Without ADHD, Excluding Children under Assessment for ADHD, All Ages

9

Model n χ2 df p χ2 df p CFIRMSEA (90% CI) SRMR AIC

ADHD (excluding children under assessment) 198 152.95 - - - - - - - - -

No ADHD 550 165.47 - - - - - - - - -

Configural Invariance 748 318.42 124 <.001 - - - 0.889 0.065[0.056 0.074] 0.055 62360

Metric Invariance 748 328.75 134 <.001 8.74 10 .557 0.889 0.062[0.054 0.071] 0.059 62350


Significant Relationships Between Cognitive Factor Scores and Standard Scores on Reading and Maths for Children With and Without ADHD, All Ages

10

Cognitive Dimensions for Children Aged 8 years and Over


11

Correlations between Cognitive and Learning Measures for Children 8 years and Older

Variable 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

1. Alliteration

2. Rapid Naming .21**

3. Nonword Repetition

.44** .15**

4. Visual Scanning

.16** .33** .07

5. Motor Speed .07 .20** -.02 .40**

6. Digit Recall .40** .17** .52** .18** .13**

7. Dot Matrix .26** .21** .15** .25** .27** .30**

8. Backward Digit Recall

.35** .26** .31** .21** .20** .46** .44**

9. Mr X .22** .16** .17** .21** .18** .26** .38** .34**

10. Following Instructions

.20** .22** .26** .16** .12* .23** .29** .25** .26**

11. Delayed Recall

.36** .14** .28** .13** .15** .28** .21** .25** .27** .22**

12. SRT .21** .18** .19** .14** .03 .07 .23** .16** .18** .18** .13**

13. Vigil .22** .17** .18** .18** .13** .14** .19** .18** .20** .20** .16** .27**

14. Cancellation .17** .30** .14** .43** .42** .17** .28** .23** .20** .18** .22** .18** .19**

15. Switching (RBBS)

.07 .28** .12* .20** .24** .06 .18** .16** .20** .16** .18** .15** .10* .36**

16. Matrix Reasoning

.33** .11* .28** .13** .23** .29** .38** .35** .35** .31** .30** .17** .23** .24** .09*

12

17. Planning .16** .15** .10 .19** .09 .11* .19** .18** .25** .21** .20** .07 .17** .22** .16** .27**

18. Number Letter Switching

.27** .29** .21** .36** .28** .32** .31** .33** .29** .32** .24** .08 .16** .34** .33** .36** .26**

19. Reading .50** .29** .51** .14** .12** .43** .26** .40** .23** .23** .31** .20** .17** .20** .10* .36** .15** .28**

20. Spelling .40** .25** .41** .13** .04 .36** .21** .40** .19** .18** .24** .15** .16** .20** .07 .29** .05 .24** .81**

21. Maths .37** .23** .36** .21** .19** .37** .37** .44** .39** .30** .25** .23** .30** .31** .18** .55** .23** .42** .48** .46**

Note.RBBS = Reds, Blues, Bags and Shoes.

* p < .05. ** p < .01

Exploratory Factor Analysis (EFA)

A parallel analysis yielded a five-factor solution. An initial EFA testing this solution revealed the model was a good fit to the data, χ2(73) = 131.26

RMSEA=0.038 (90% confidence interval [CI] =0.027, 0.048), CFI =0.972, RMSR = 0.03. However, close inspection highlighted several issues with this

13

model. These included multiple cross-loadings and factors with single variable loadings (see Supplementary Table 9). In addition, the fourth and fifth factors

accounted for only 5% and 3% of the total variance, respectively.

To test whether the parallel analysis was overfitting the data, a four factor EFA was run. This solution was a good fit, χ2(87) = 182.23 , RMSEA=0.045

(90% confidence interval [CI]=0.035,0.053), CFI =0.956, RMSR = 0.03 (see Supplementary Table 10). The third factor in this model was indicated by a single

variable (e.g. Digit Recall). Factors being indicated by single variables can be a sign of overfitting. In addition, there is little previous evidence to suggest that

Digit Recall will form a latent construct on its own when entered into analyses with other phonological and verbal memory measures.

A three-factor solution was tested next to determine whether a more theoretically interpretable solution could be found with factors identified by more

than one variable. This model did not fit the data as well as the four or five factor models, χ2(102) = 262.65, RMSEA = 0.053 (90% confidence interval

[CI]=0.045 ,0.061), CFI =0.925, RMSR = 0.04. There were two weak paths (factor loadings<.25), the SRT and Vigilance tasks. These variables were removed

and the model was re-estimated. The re-estimated model fit the data to an acceptable level, χ2(75) = 191.84 , RMSEA = 0.053 (90% confidence interval

[CI]=0.043 ,0.062), CFI =0.942, RMSR = 0.04, without any cross-loadings (see Supplementary Table 9).

The results from the EFAs were used to fit CFAs for each model. To test which model provided the best fit to the data, the five- and four-factor models

were both compared to the three-factor model. The five-factor model, χ2 (126) =297.66 RMSEA= 0.049 (90% confidence interval [CI] = 0.042, 0.056), CFI =

0.909 , SRMR = 0.051, provided a significantly better fit than the three-factor model, χ2 (132) = 316.84, RMSEA= 0.050 (90% confidence interval [CI] =

0.043, 0.057), CFI = 0.902 , SRMR = 0.052, χ2 = 18.57, df = 6, p =.005. The four factor model, χ2 (130) = 361.78, RMSEA= 0.056 (90% confidence

interval [CI] = 0.049, 0.063), CFI = 0.877 , SRMR = 0.056 provided an acceptable fit to the data, with the exception of the CFI. There was no statistically

significant difference in model fit between the four- and three-factor models, χ2 = -50.96, df = 2, p = 1.00. However, the three-factor solution yielded lower

AIC and BIC values than the 4-factor solution (four-factor AIC, 59166, BIC, 59422; three-factor AIC, 59117, BIC 59364), indicating a marginally better fit to

14

the data for the three factor model. So, overall the five-factor model was the best fit but it was overfitting the data, as indicated by multiple factors having single

indicators. The three factor model was marginally better than the four factor model, and all factors were indicated by more than one variable. For this reason,

the three factor model was selected as the best fit to the data.


Five- vs Four- vs Three- Factor Exploratory Factor Analysis for Children 8 years and Older

15

Five- Factor Model Four-Factor Model Three-Factor Model

Factor Loadings F1 F2 F3 F4 F5 F1 F2 F3 F4 F1 F2 F3Alliteration 0 0.43 0.2 0.15 -0.06 0.25 0.02 0.25 0.31 0.23 0.02 0.45Rapid Naming 0.45 0.11 -0.05 0.14 -0.04 0 0.47 0.03 0.14 -0.02 0.47 0.12Nonword Repetition -0.05 0.7 0.05 0.12 -0.22 0.01 0 0.46 0.49 -0.06 0.01 0.83Visual Scanning 0.66 0.04 -0.1 0.04 0.06 -0.04 0.65 0.06 -0.06 -0.01 0.64 -0.02Motor Speed 0.57 -0.08 0.05 -0.11 0.14 0.07 0.55 0.03 -0.24 0.13 0.53 -0.17Digit Recall 0.06 0.78 -0.02 -0.1 0.15 0.05 0.03 0.82 -0.01 0.24 0.01 0.53Dot Matrix 0.11 0.04 0.32 0.17 0.43 0.6 0.07 0.06 -0.14 0.64 0.07 -0.08Backward Digit Recall 0.09 0.37 0.19 0.06 0.3 0.42 0.05 0.33 -0.05 0.50 0.04 0.20Mr X 0.04 0.01 0.45 0.08 0.17 0.59 0.01 -0.01 -0.02 0.59 0.01 -0.04Following Instructions 0.07 0.08 0.38 0.1 -0.04 0.39 0.07 -0.01 0.17 0.33 0.09 0.14Delayed Recall 0.07 0.19 0.36 0.01 -0.12 0.29 0.08 0.09 0.21 0.27 0.09 0.23Simple Reaction Time 0.04 -0.02 -0.01 0.7 0.04 0.27 0.09 -0.17 0.31 - - -Vigil 0.1 0.03 0.17 0.28 -0.02 0.27 0.13 -0.09 0.23 - - -Cancellation 0.66 0 0.04 0.06 -0.07 0 0.69 -0.03 0.07 -0.04 0.70 0.04Switching (RBBS) 0.45 -0.06 0.12 0.06 -0.16 0.03 0.48 -0.12 0.15 -0.04 0.49 0.04Matrix Reasoning -0.05 0.06 0.63 0.02 0.07 0.64 -0.05 0.02 0.07 0.61 -0.04 0.08Planning 0.12 -0.1 0.46 -0.04 -0.1 0.35 0.12 -0.12 0.08 0.30 0.13 -0.03Trails 0.39 0.13 0.34 -0.13 -0.04 0.25 0.37 0.13 0.01 0.28 0.37 0.09

Note. The paths for SRT and vigil were weak (<.25) and therefore removed from the three-factor model. RBBS= Reds, Blues, Bags and Shoes.

Cognitive Dimensions for Children Aged 8 years and Older - Summary

16

An EFA identified a three-factor solution. Measures of visuo-spatial and verbal WM, visual STM, episodic memory, Matrix Reasoning and planning

loaded most highly on Factor 1. These tasks all required executive functions. The speeded and timed tasks, including Rapid Automated Naming, Visual

Scanning, Motor Speed, Cancellation, and switching loaded on to the second factor. The third factor was most strongly associated with measures requiring

phonological processing: Alliteration, nonword repetition, and Digit Recall. The three factors were therefore labelled executive function, processing speed and

phonological processing, respectively.

Comparison of Children 5-7 years and Children 8 years and Older

Measurement invariance was conducted to test whether the three factor model identified for the whole sample (see main manuscript) fit both children

aged 7 and under, and 8 and over. This was necessary as the analyses reported above for the 8 and older sample include a greater number of measures than

those included the primary analyses for the whole sample. The measurement invariance reported here includes only measures administered to all children (see

Supplementary Table 10 for outcomes).


17

Fit Statistics for Measurement Invariance Comparing Children 5-7 years and 8 years and Older

Model n χ2 df p χ2 df p CFI RMSEA SRMR AIC5-7 years 239 104.126 - - - - - - - - -

8 years 566 200.561 - - - - - - - - -


[0.052-0.069]0.050 66986

Metric Invariance 805 344.429 134 <.001 40.96 10 <.001 0.889 0.062[0.054-0.071] 0.060 67005

Metric Invariancea 805 314.79 133 <.00110.14

79 .339 0.904 0.058

[0.050-0.067] 0.053 66978

Note. a. Free estimation of loadings of Backwards Digit Recall on the executive factor.

* p < .05.

Links between Cognition and Learning for Children Aged 8 years and Older

18


Correlations between Cognitive Factor Scores and Learning Outcomes for Children 8 years and Older

Variable 1 2 3 4 5

1. Executive Function 2. Speed .77**3. Phonological Processing .86** .53**4. Reading .53** .36** .60**5. Spelling .45** .30** .50** .81**6. Maths .63** .48** .55** .48** .46**

Note. * p < .05, ** ip < .01.


19

Relationships between Cognitive Factor Scores and Learning Measures for Children 8 years and Older

Note. Distributions are displayed on the diagonal, Pearson correlations in the upper triangle and simple linear regressions in the lower triangle.


20

Path Models Predicting Learning Outcomes from Cognitive Factors for Children 8 years and Older

Path Estimate SE z pFully Standardized

EstimateReading

Phonological Processing 0.761 0.095 8.010 <.001** 0.598Speed 0.695 0.528 1.32 .189 0.079Executive Functions -0.08 0.188 -0.426 .670 -0.041R2 0.367

Spelling

Phonological Processing 0.509 0.088 5.766 <.001** 0.471Speed 0.468 0.493 0.950 .342 0.062Executive Functions -0.012 0.181 -0.066 .947 -0.007R2 0.250

Maths

Phonological Processing 0.044 0.092 0.482 .630 0.034Speed 0.061 0.565 0.108 .914 0.007Executive Functions 1.189 0.20 5.94 <.001** 0.593R2 0.394

Note. * p < .05, ** p < .01.

21

Children With and Without ADHD Aged 8 years and Older


Descriptive Statistics for Children with ADHD and Possible ADHD, 8 years and Older

Children with ADHD (N=156) Children with possible ADHD (N=36) Group ComparisonsMeasurement n min max M SD SE n min max M SD SE t p dPhonological ProcessingAlliteration 152 70 101 92.47 9.69 0.79 35 70 101 94.71 8.68 1.47 -1.26 .210 -0.237Rapid Naming 150 0 131 92.26 15.8 1.29 34 69 121 90.03 13.98 2.4 0.76 .449 0.145Non-word Repetition 133 45 123 80.49 23 1.99 33 45 119 84.55 21.84 3.8 -0.92 .361 -0.179Processing SpeedVisual Scanning 141 1 16 8.92 3.65 0.31 34 1 16 9.88 3.98 0.68 -1.35 .178 -0.26Motor Speed 141 1 14 10.33 2.66 0.22 34 4 14 11.18 2.47 0.42 -1.70 .092 -0.326Simple Reaction Rime 139 1 19 8.37 4.42 0.38 32 1 16 7.09 3.68 0.65 1.52 .131 0.3WM/STMDigit Recall 156 60 149 90.37 16.2 1.3 36 64 128 92.78 14.73 2.46 -0.82 .416 -0.152Dot Matrix 156 56 135 89.78 16.3 1.31 36 62 125 87.41 15.33 2.56 0.80 .427 0.148Backward Digit Recall 155 70 137 90.96 11.93 0.96 36 78 119 94.72 10.04 1.67 -1.75 .082 -0.326Mr X 156 62 131 94.55 14.48 1.16 35 72 134 101.05 14.34 2.42 -2.40 .017* -0.452Following Instructions 150 -8.81 18.95 0.27 4.1 0.33 34 -7.47 7.87 0.55 3.46 0.59 -0.37 .711 -0.071Episodic MemoryDelayed Recall 149 1 17 7.42 3.28 0.27 35 2 13 7.97 3.14 0.53 -0.91 .365 -0.171Executive FunctionVigil 141 3 15 8.35 3.35 0.28 33 4 16 9.12 3.8 0.66 -1.15 .250 -0.224Cancellation 152 1 19 10.55 3.58 0.29 35 2 19 11.66 3.26 0.55 -1.68 .095 -0.317Switching (RBBS) 140 1 16 7.91 3.45 0.29 32 1 15 8.91 3.61 0.64 -1.47 .145 -0.289Planning (Towers) 137 3 16 9.58 2.66 0.23 34 4 16 9.62 2.47 0.42 -0.07 .947 -0.013Number Letter Switching 129 1 15 6.7 3.92 0.35 29 2 14 7.69 3.47 0.64 -1.26 .211 -0.26

22

Nonverbal ReasoningMatrix Reasoning 156 23 72 42.34 10.18 0.82 36 25 73 47.86 11.15 1.86 -2.88 .004** -0.535Learning MeasuresReading 151 49 129 88.75 16.61 1.35 35 71 118 94.71 12.66 2.14 -1.99 .048* -0.376Spelling 151 48 131 84.4 15.27 1.24 35 69 118 90.63 13.06 2.21 -2.23 .027* -0.42Maths 155 45 156 84.1 18.62 1.5 34 70 128 94.62 17.19 2.95 -3.02 .003** -0.575

Note. Residual scores were calculated for the Following Instructions task.

* p < .05, ** p < .01.

23

Tests of measurement invariance revealed that, with the exception of CFI values, the three-factor cognitive model was an acceptable fit for both groups,

meeting criteria for configural invariance (Supplementary Table 14). Constraints on the factor loadings were applied across groups. There was no significant

deterioration of model fit, χ2 = 15.64, df =13, p =.269, indicating metric invariance was met, even when children with possible ADHD were removed from

the analysis (Supplementary Table 15).


Fit Statistics for Measurement Invariance Comparing Children With and Without ADHD, 8 years and Older

Model n χ2 df p χ2

df p CFI

RMSEA

(90% CI) SRMR AIC

ADHD 192

213.2

4 - - - - - - - - -

No ADHD 374

178.4

8 - - - - - - - - -

Configural Invariance 566

391.7

1 202 <.001 - - - 0.894

0.058

[0.049, 0.066] 0.063 53499

Metric Invariance 566

408.9

0 215 <.001 15.64 13 0.269 0.891

0.056

[0.048, 0.065] 0.069 53490

24


Fit Statistics for Measurement Invariance in Children With and Without ADHD, Excluding Children under Assessment for ADHD, 8 years and Older


ADHD (excluding children under assessment)

156 205.905 - - - - - - - - -

No ADHD374 178.475 - - - - - - - - -

Configural Invariance 530 384.379 202<.00

1 - - - 0.8900.058

[0.049 0.067] 0.064 49963

Metric Invariance 530 402.856 215<.00

1 16.865 13 0.20554 0.8870.057

[0.049 0.066] 0.070 49956

Equality-constrained and freely-estimated path models were used to compare pathways between the three cognitive dimensions and each learning

outcome across those with and without ADHD. The constrained models fit better than freely estimated models for all learning outcomes (reading χ2(3) = 3.64,

p = 0.303, spelling χ2(3) = 3.49, p = 0.323, maths χ2(3) =4.90, p = 0.180, were the same across groups. For both groups, phonological processing was

significantly associated with reading and spelling, and executive function with maths (see Table 7).

Tasks

Where possible, age-standardised tests were selected that spanned the full age range of the sample. Measures of phonological processing; Rapid

Automated Naming (RAN), Alliteration and CNRep (nonword repetition) were not standardised for the full age range of the sample, limited to children aged up

to 11years for RAN and Alliteration, and to 9 years for CNRep. Closest-age matching was used to derive age-standardised scores for children outside the age-

25

normed reference frame (e.g. for those aged 11 and over, norms for 11 year olds were used). Density plots and skewness values for children for whom closest-

age matching was used, and for children in the next age band down (e.g. children aged 11+, and children aged 9-10 years for RAN and Alliteration; children

aged 9+, and aged 7-8, for CNRep) are provided in Supplementary Figures 5-7. Scores were distributed similarly for both age groups for all three tasks, and the

measures of skew were comparable for both children with and without age-normed scores. All measures of skew were below 1 and therefore within an

acceptable range (Tabachnik & Fidell, 2007).


Alliteration Scores for Children Aged 11+ (skew -.79) and those Aged 9-10 years (skew -.63)

Note. Dotted lines represent group means.

26


Rapid Automated Naming Scores for Children Aged 11+ (skew .27) and those Aged 9-10 years (skew .25)

Note. Dotted lines represent group means.

27


CNRep Scores for Children Aged 9+ (skew -.45) and those Aged 7-8 years (skew -.20)

Note. Dotted lines represents group means.

Additional Supplementary Analyses Described in the Pre-Registration

The pre-registered analysis plan stated that measurement invariance would be used to determine whether the winning cognitive dimensions model fits

subgroups within the sample. The original plan was to test whether the winning model fitted: a) children with and without an ADHD diagnosis; b) children with

standard scores <86 on the learning measures compared to those with scores >86 (poor vs typical learners; c) children with and without an ADHD diagnosis

28

within the subgroup of poor learners. On re-consideration the decision was made prior to the commencement of analysis to remove the comparisons made with

poor learners in steps b) and c) from the main manuscript, but to report them here in the Supplementary Materials for completeness.

Comparison of Children With and Without Learning Problems, and Children with Learning Problems accompanied by ADHD or Not, All Ages

Poor vs Typical Learners, All Ages

Children from the whole sample were categorised as either “poor” (scores <86 on either spelling, reading, or maths) “typical learners” (scores >86 on all

learning measures). A multi-group measurement invariance model was fit to the subgroups of poor (N=601) and typical (N=204) learners (see Supplementary

Supplementary Table 16). Metric invariance was tested across groups by imposing equality constraints on the factor loadings. This analysis revealed that the

freely-estimated model, without any constraints, outperformed the constrained model, χ2 = 19.95, df = 10, p = .03, and therefore that metric invariance was

not supported. The violation of metric invariance suggested that relationship between the cognitive tasks and dimensions differed across groups. Inspection of

the factor loadings revealed that this violation resulted from many small differences rather than one alone. Specifically, the loadings of matrix reasoning and

backwards digit recall onto executive functions, and following instructions onto speed revealed the largest discrepancies between groups (see Supplementary

Table 17). Freely estimating each of these parameters significantly improved the model fit and achieved partial metric invariance, (matrix reasoning, χ2

=12.85, df =9, p = .170, Backwards Digit Recall, χ2 =14.60, df =9, p = .102, following instructions, χ2 =17.19, df =9, p = .05. It was not possible to test

whether links between the three cognitive dimensions and learning were comparable across groups because metric/weak invariance was violated, and the factor

loadings differed significantly between children with and without learning difficulties, as revealed by the tests of measurement invariance. This significant

difference between groups precluded any meaningful comparison of associations between cognitive dimensions and learning outcomes, and therefore these

analyses were not conducted.

29


Fit Statistics for Measurement Invariance Assessment in Children with and without Learning Problems, All Ages


Learning Problems 601 201.25 - - - - - - - - -No Learning Problems 204 110.63 - - - - - - - - -

Configural Invariance 789 311.88 124 <.001 - - - 0.871

0.061[0.053, 0.070] 0.056 66969

Metric Invariance 789 333.57 134 <.001 19.95 10 .03 0.863

0.061[0.053, 0.069] 0.060 66970

30


Latent Variable Loadings for Three-Factor Model in Subgroups of Children With and Without Learning Problems (Free Model)

Fully Standardised Estimate Path

Group

Learning Problems (N=601) No Learning Problems (N=204)Phonological =~

Digit Recall 0.676 0.84Nonword Repetition 0.63 0.623Delayed Recall 0.457 0.291

Speed =~Alliteration 0.588 0.452Simple Reaction Time 0.468 0.364Vigil 0.474 0.435Rapid Naming 0.432 0.421Cancellation 0.454 0.367Following Instructions 0.476 0.484

Executive =~Dot Matrix 0.573 0.602MrX 0.52 0.512Matrix Reasoning 0.521 0.335Backward Digit Recall 0.626 0.717

Note. Values represent fully standardized estimates. Phonological = Phonological Processing

31

Poor Learners With and Without Co-Occurring ADHD, All Ages

Children within the “poor learner” group (scores <86 on at least one measure of learning (e.g. spelling, reading, or maths)) were split according to

whether they had co-occurring ADHD (N=179) or not (N=422). Comparisons across these groups revealed that there were no statistical differences between

models of configural and weak constraints, χ2 = 6.09, df = 10, p =.808. These results indicated that there was weak invariance across groups; overall model

structure and factor loadings did not differ significantly between children with learning difficulties and co-occurring ADHD or not (Supplementary Table 18).


Fit Statistics for Measurement Invariance Assessment in Children with Learning Problems Accompanied by ADHD or not, All Ages

Model n χ2 df p χ2 df p CFIRMSEA

(90% CI) SRMR AIC

Learning Problems + ADHD 179137.1

8 - - - - - - - - -

Learning Problems + no ADHD 422155.5

9 - - - - - - - - -

Configural Invariance 601292.7

8 124 <.001 - - - 0.8560.067

[0.057, 0.077 0.062 49740

Metric Invariance 601300.2

8 134 <.001 6.10 100.807

8 0.8580.065

0.055, 0.074 ] 0.064 49728

The links between the three cognitive dimensions of phonological processing, executive function and processing speed and each learning outcome were

compared across groups. The freely estimated model fit better than the constrained model for reading only, χ2 (3)= 10.10, p = .020. Inspection of parameter

32

estimates indicated that reading was predicted most strongly by speed (standardized beta: 0.629), phonological (standardized beta: 0.339) and executive factors

(standardized beta: -0.469) in children with learning difficulties and co-occurring ADHD. In children with learning difficulties alone, however, reading was

associated only with phonological processing (standardized beta: 0.526).

When paths from the cognitive factors to spelling and maths were constrained to be equal, the models did not differ significantly from those in which

the paths were estimated freely (spelling, χ2 (3) = 6.01, p = .110, maths, χ2 (3) =2.23, p = .523). This indicated that the predictors for spelling and maths did

not differ significantly across groups. Spelling was associated with phonological processing, and maths with executive functions (see Supplementary Table 19).

33


Path Models Predicting Learning Outcomes from Cognition in Children with Learning Difficulties and ADHD or not, All Ages

Children with Learning Difficulties and no ADHD (n=422)

Children with Learning Difficulties and ADHD

(n=179)

Estimates - Free Models Estimate SE z pFully

Standardized Estimate

Estimate SE z pFully


ReadingReading ~ Phono 0.858 0.157 5.457 <.001** 0.526 0.59 0.204 2.897 0.004** 0.339Reading ~ Speed 0.478 0.285 1.679 0.093 0.174 1.971 0.409 4.815 <.001** 0.629

Reading ~ Executive -0.462 0.282-

1.637 0.102 -0.225 -1.092 0.377 -2.899 0.005** -0.469R2 0.222 0.262

Estimates - Constrained Models Estimate SE z pFully




SpellingSpelling ~ Phono 0.399 0.115 3.477 <.001** 0.321 0.399 0.115 3.477 <.001** 0.294Spelling ~ Speed 0.195 0.191 1.017 0.309 0.093 0.195 0.191 1.017 0.309 0.08

Spelling ~ Executive -0.145 0.194-

0.746 0.455 -0.093 -0.145 0.194 -0.746 0.455 -0.08R2 0.099 0.083Maths

Maths ~ Phono -0.182 0.105-

1.736 0.083 -0.119 -0.182 0.105 -1.736 0.083 -0.118Maths ~ Speed 0.089 0.199 0.447 0.655 0.035 0.089 0.199 0.447 0.655 0.032Maths ~ Executive 1.171 0.215 5.44 <.001** 0.61 1.171 0.215 5.44 <.001** 0.568R2 0.291 0.252

Note. Phono=Phonological Processing.

*p < .05. ** p < .01.

34

Comparison of Children With and Without Learning Problems, and Children with Learning Problems Accompanied by ADHD or not, Children 8

years and Older

Poor vs Typical Learners, Children aged 8 years and older

The subgroup of children aged eight years and older were grouped into “poor” (scores <86 on either spelling, reading, maths N=434) or “typical

learners” (scores >86 on all learning measures, N=132). Tests for configural invariance revealed that the three-factor model, yielding the best account of the

data for the whole eight plus sample, provided an acceptable fit to the data (Supplementary Table 20). With the exception of the CFI, all indices were within the

specified limits (e.g., RMSEA 0.6-0.8; SRMR 0.6-0.8), indicating that the cognitive measures loaded on the same three factors in children with and without

learning problems. When the factor loadings were constrained, the fit statistics revealed that metric invariance was supported, χ2 = 17.70, df = 13, p = .17.

This indicates that factor loadings were the same across groups. Given metric invariance was established, the links between cognition and learning were

compared across groups in multi-group regressions (path models).

35


Fit Statistics for Measurement Invariance Assessment in Children With and Without Learning Problems for the Sample Aged 8 years and Older

Model n χ2 df p χ2 df p CFI RMSEA SRMR AIC

Learning Problems 434 214.46 - - - - - - - - -No Learning Problems 132 166.23 - - - - - - - - -

Configural Invariance 566 380.69 202 <.001 - - - 0.882

0.056[0.047 0.064] 0.062 53325

Metric Invariance 566 400.86 215 <.001 17.70 13 0.169 0.877

0.055[0.047 0.064] 0.067 53319

Path models (multigroup regressions) revealed that the freely estimated models for reading, spelling and maths fit better than the constrained models,

(reading χ2(3) = 12.239, p = 0.007, spelling χ2(3)= 9.104, p = 0.028, maths χ2(3) = 8.858, p = 0.03), indicating that the pathways between cognition and

learning were not the same for children with and without learning difficulties (Supplementary Table 21). For both groups, phonological processing significantly

predicted reading, but for children with learning problems, this prediction was stronger. In contrast, phonological processing and executive functions

significantly predicted spelling for children with learning difficulties, but not for children without learning difficulties. Finally, executive functions predicted

maths in both groups of children, but for children without learning difficulties, this prediction was stronger.

36


Path Models Predicting Learning Outcomes from Cognition in Children With and Without Learning Difficulties, 8 years and Older

Children with Learning Difficulties (n=434) Children without Learning Difficulties (n=132)

Estimates - Free Models Estimate SE z pFully




ReadingReading ~ Phono 0.759 0.104 7.278 <.001** 0.612 0.3 0.112 2.682 0.007** 0.355Reading ~ Speed 0.987 0.603 1.638 0.101 0.115 -0.28 0.601 -0.465 0.642 -0.057Reading ~ Executive -0.346 0.207 -1.673 0.094 -0.177 0.198 0.228 0.869 0.385 0.171R2 0.274 0.224SpellingSpelling ~ Phono 0.49 0.091 5.409 <.001** 0.524 0.032 0.129 0.248 0.804 0.038Spelling ~ Speed 0.962 0.495 1.945 0.052 0.148 -1.047 0.739 -1.416 0.157 -0.212Spelling ~ Executive -0.393 0.177 -2.227 0.026* -0.267 0.461 0.274 1.682 0.093 0.394R2 0.148 0.09MathsMaths ~ Phono -0.117 0.088 -1.329 0.184 -0.104 0.112 0.17 0.662 0.508 0.096Maths ~ Speed 0.554 0.535 1.036 0.3 0.071 -1.523 0.939 -1.622 0.105 -0.224Maths ~ Executive 0.994 0.216 4.603 <.001** 0.559 1.028 0.319 3.22 0.001** 0.637R2 0.282 0.323

Note. Phono=Phonological Processing.

*p < .05. ** p < .01.

37

Poor learners with and without co-occurring ADHD

The poor learner group were selected from the subsample aged eight and over, and then split into those with (N=136) and without ADHD (N=298).

Metric invariance tests revealed that the three-factor model was acceptable, although the CFI value indicated a poor model fit (see Supplementary Table 22).

Change-in-fit indices revealed that metric, χ2 =10.37, df =13, p =0.664 invariance was supported.


Fit Statistics for Measurement Invariance Assessment in Children with Learning Problems With and Without ADHD for the Sample, 8 years and Older

Model n χ2 df p χ2df p CFI

RMSEA[90% CI) SRMR AIC

Learning Problems + ADHD 136 187.41 - - - - - - - - -Learning Problems + no ADHD 298 176.45 - - - - - - - - -


[0.051 0.071] 0.072 40567

Metric Invariance 434 375.52 215 <.001 10.37 13 0.664 0.8600.059

[0.049 0.068] 0.075 40553 Note. aFree estimation of Visual Scanning intercepts.

Path models (multigroup regressions) revealed that all constrained models fit better than the freely estimated models, (reading χ2(3) = 3.11, p =0.375,

spelling χ2(3)= 3.16, p =0.368, maths χ2(3) = 0.54, p =0.909 ), indicating that the pathways between cognition and learning were the same for children with

learning difficulties with and without occurring ADHD (Supplementary Table 23). Phonological processing significantly predicted reading and spelling, and

executive function predicted maths. Speed additionally predicted spelling.

38


Path Models Predicting Learning Outcomes from Cognition in Children with Learning Difficulties With or Without Co-Occurring ADHD, 8 years and older

Children with Learning Difficulties and No ADHD

(n= 298)

Children with Learning Difficulties and ADHD

(n=136)

Estimates - Constrained ModelsEstimate SE z p Fully


Estimate SE z p Fully Standardized

EstimateReading

Reading ~ Phono 0.7630.10

5 7.285 <.001** 0.619 0.763 0.105 7.285 <.001** 0.613

Reading ~ Speed 0.9160.60

5 1.515 0.13 0.105 0.916 0.605 1.515 0.13 0.111

Reading ~ Executive -0.3350.20

7 -1.618 0.106 -0.175 -0.335 0.207 -1.618 0.106 -0.164R2 0.277 0.284Spelling

Spelling ~ Phono 0.4970.08

8 5.651 <.001** 0.56 0.497 0.088 5.651 <.001** 0.482

Spelling ~ Speed 0.9010.50

4 1.786 0.074 0.143 0.901 0.504 1.786 0.074 0.132

Spelling ~ Executive -0.4050.16

9 -2.395 0.017* -0.294 -0.405 0.169 -2.395 0.017* -0.239R2 0.156 0.130Maths

Maths ~ Phono -0.1180.08

8 -1.338 0.181 -0.103 -0.118 0.088 -1.338 0.181 -0.106

Maths ~ Speed 0.5620.53

8 1.044 0.296 0.069 0.562 0.538 1.044 0.296 0.077

Maths ~ Executive 0.9950.21

5 4.619 <.001** 0.563 0.995 0.215 4.619 <.001** 0.549R2 0.283 0.281

Note. Phono=Phonological Processing. 39

*p < .05. ** p < .01.

40

supp.apa.org · web viewa parallel analysis yielded a five-factor solution. an initial efa...

Documents