Supplemental Material: Heterogeneity of executive function revealed by a functional random forest approach across ADHD and ASD

1. Methods

1.1 Participant recruitment and diagnostics

ASD participants were recruited via community outreach and mailings, and from OHSU's Child

Development and Rehabilitation Center (CDRC.) ASD participants were phone screened for

basic eligibility criteria prior to their first visit - where both parent and child completed initial

diagnostic testing and questionnaires. After the assessment visit, participants received a

consensus diagnosis from a team of two licensed psychologists and a psychiatrist using research

reliable clinical interviews: Autism Diagnostic Observation Schedule (ADOS-2) (C Lord et al.,

2012) and the Autism Diagnostic Interview (ADI-R) (Catherine Lord et al., 1994), as well as

parent and teacher answered questionnaires including the Social Responsiveness Scale (SRS-2)

(Constantino & Gruber, 2005), Children’s Communication Checklist (CCC-2) (Bishop, 2013),

and a developmental history.

ADHD and typically developing (TD) participants were recruited in a similar fashion through a

parallel study (Karalunas et al., 2018) and were diagnosed with (or without) ADHD and other

comorbidities by a separate clinical team of a similar profile using the Kiddie Schedule of

Affective Disorders and Schizophrenia (K-SADS) (Kaufman et al., 1997) parent interview, as

well as parent and teacher answered questionnaires including the ADHD Rating Scale (ADHD-

RS) (DuPaul et al., 2016), Conners-3 (Conners et al., 2011), Strengths and Difficulties

Questionnaire (SDQ) (Goodman, 1997), Children’s Depression Inventory (CDI) (Kovacs, 1985),

1

123456789

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

and the Multidimensional Anxiety Scale for Children (MASC-2) (March, 2004). All ASD

participants completed the same diagnostic process to assess for ADHD and other comorbidities

and 36 of the ASD participants also met criteria for a comorbid ADHD diagnosis (age mean =

12.3, SD = 2.3). 19 of the ASD participants received a “subthreshold-ADHD” diagnosis -

meaning that they met 5 but not 6 symptoms of inattention or hyperactivity, just shy of the

diagnostic threshold by DSM-IV and DSM-5. Additionally, as a final way to minimize false

positive cases, only participants who met DSM-IV and DSM-5 criteria for ADHD at every

participating study year were included as participants in the ADHD cohort. Similarly, TD

participants were included if they had a stable TD diagnosis at every participating study year.

ASD participants were excluded if they were taking long-acting psychoactive medication, had

genetic abnormalities, neurological impairments, major medical issues or physical disabilities,

significantly impairing Axis I comorbidities (i.e., bipolar disorder, schizophrenia), closed head

injury, seizure disorders, or if they had an estimated IQ <70. ADHD and TD participants

followed the same exclusion criteria with the additional exclusion of an ASD diagnosis, and no-

ADHD for the TD participants. All participants were required to be washed-out of any stimulant

medication for 24-48 hours (depending on the half-life and preparation) for all

neuropsychological and behavioral testing and MRI visits.

1.2 Data Collection Process

After the initial screening visit and diagnostic consensus, ASD participants returned for three

additional visits where the majority of the neuropsychological task data, as described in the

section below, were collected and the participants underwent an MRI. Study data for the single

year of participation were included for ASD participants.

2

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

ADHD and TD participants completed their EF testing and MRI at a companion study of a

longitudinal design (see Karalunas et al). During the first year of the ASD study, companion

study participants were invited to complete a single visit with the ASD study where remaining

data were collected to match the ASD participant data.

1.3 Data matching and further exclusions

ASD participant data were included for a single year of participation. The mean average of time

between the first and last visit was 3 months. For the ADHD and TD participants – their ASD

study visit date was considered the initial time-point and data collected through the companion

study were matched as closely as possible to the ASD data collection dates. As some EF and

ADHD symptoms are known to change or improve over time, any ADHD or TD participant with

ASD study data that exceeded a 9 month cutoff from the companion study were not included in

our participant list.

1.4 Individual task descriptions and review

A total of 43 variables from multiple behavioral tasks and one parent answered questionnaire

were used as inputs in the FRF models. The measures span multiple cognitive domains as

identified in the main manuscript, and are categorized into four broader domains including

cognitive flexibility, response inhibition, working memory, and task control. Tables 1, 2, 3, and 4

summarize the variable inputs, measures they are derived from, and the how the score was

represented.

1.4.1 Verbal Fluency: Cognitive flexibility

The NEPSY verbal fluency task (Korkman et al., 2007)measures a participant’s ability to recall

and list out words in 2 categories (semantic: animals and food) and recall words beginning with a

letter (phonemic: S and F) over a 1 minute span for each item. Including both categorical and

3

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

letter fluency allows for the testing of semantic and phonemic word generation and retrieval. The

verbal fluency task was audio recorded and reviewed by both the administrator and a secondary

tester to verify the accuracy of administration and scoring. Variables included in the model were

the semantic, phonemic, and contrast scores – all of which were scaled.

The majority of prior research indicates that those with ASD tend to perform worse on semantic

word generation than their TD peers (H M Geurts et al., 2004; Spek et al., 2009; Verté et al.,

2006). Still, some researchers have shown equal numbers of total semantic word production for

ASD and TD children and adolescents (ages 6-23) (Begeer et al., 2014). Another group found

delays in performance on letter fluency for ADHD adolescents as compared to TD (Hurks et al.,

2004).

In one study, male ASD participants performed a VF task while undergoing fMRI (Kenworthy et

al., 2013). They found reduced activity in regions associated with executive control compared to

TD, despite no significant differences in word generativity. The ages of the participants in this

study were similar, but had mean of 16(ASD) and 17(TD) years old, putting both groups over the

threshold where one would expect ASD participants to perform more similarly to their TD peers.

1.4.2 Behavior Rating Inventory of Executive Function (BRIEF): Response inhibition,

cognitive flexibility, working memory, task control

The BRIEF (Gioia et al., 2001) is a parent answered questionnaire that covers a broad range of

EF as exhibited by the child in everyday life. For example, an item from the emotional control

module states “Has explosive angry outbursts” and the parent is asked to rate the truthfulness of

the statement, as it pertains to their child, as never (1), sometimes (2), or often (3). The scored

categories include inhibition, shifting, emotional control, working memory, planning and

organizing, organization of materials, monitoring, behavioral regulation, and metacognition.

4

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

Parents answered the questionnaire on a computer or tablet. T-Scores from all domains were

included in the model.

The newness of the BRIEF (2000) may contribute to the limited amount of research. Children

with ADHD have shown impaired ratings on the BRIEF, and the BRIEF itself is highly

correlated with other questionnaire based measures of EF (Mahone et al., 2002). One group

examining ASD and TD participants found that by applying a graph theory metric derived from

task and rest states, they could successfully predict a subject’s BRIEF metacognition index based

on changes within the frontoparietal, salience, and subcortical networks (Lynch et al., 2017). As

previously mentioned in the main manuscript, the BRIEF has also been used to examine the

connection between EF and everyday impairments (Gardiner & Iarocci, 2017).

1.4.3 Color Word Interference: Response inhibition

The D-KEFS (Delis et al., 2001) Color-Word Interference test is based on the Stroop procedure

and involves a participant’s ability to inhibit a learned response. In this case, we examined the

inhibitory condition in which a subject is asked to name the color of the ink that a dissonant color

word name was printed in. For example, the word “blue” might be printed in red ink and

therefore, the correct verbal response is “red.” Variables included in the model were the time to

complete the task, uncorrected errors, self-corrected errors, and total errors – all of which were

scaled.

Inhibiting a response using the stroop task has been found to be largely in-tact for those with

ASD (Adams & Jarrold, 2012). However, as with many other studies, comorbid ADHD is often

unaccounted for when comparing ASD participants to TD. Because some ASD participants may

perform worse than others based on varying levels of ADHD symptoms, we included the stroop

task in the models. Some researchers have found no differences on performance between two

5

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

ADHD “subtypes” (combined and inattentive) and their TD peers, but did find that the ADHD

subtypes were slower than the TD group which may reflect a compensatory mechanism (Nigg et

al., 2002). For this reason, we included both performance and time scores in the models.

1.4.4 Digit Span: Working memory (auditory)

The WISC-IV (Wechsler, 2003)Digit Span task measures a child’s ability to both recall and

manipulate information in short-term storage. A series of numbers are spoken aloud by the tester

and the child is then asked to recall them either in forward or backward order. The amount of

numbers in the sequences increase as the child responds correctly to each administration. For the

model, we included the backward and forward scaled scores, and percentiles for the longest digit

recalled both backward and forward.

Broadly, working memory is known to be impaired in those with ADHD, but there is conflicting

research regarding the role of auditory working memory in ASD. Some researchers found that

ASD subjects show working memory deficits similar to those seen in individuals with EF

impairments (Bennetto et al., 1996). However, another study showed that working memory may

perhaps be in-tact (Ozonoff & Strayer, 2001). It’s possible that differences in performance may

be due to varying numbers of ASD individuals with comorbid ADHD impairments in any given

cohort, which is not accounted for in these studies. Another limitation is the inclusion of only a

single working memory measure, rather than including auditory and spatial working memory in

the same analysis.

1.4.5 Trails: Cognitive flexibility

In the D-KEFS (Delis et al., 2001) trail making task the 4th condition (number-letter switching) is

the primary condition used to assess EF domains of switching, sequencing, and task control. In

condition 4, the participant is asked to trace a line from “a number (1) to a letter (A) to a number

6

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

(2) to a letter (B) and so on, in order, until they reach the end (16).” Variables included in the

model were the total time to completion, sequencing errors, set loss errors, and total errors. All of

the variables were scaled or percentiles.

Two studies showed no significant differences on versions of the trail making task between ASD

and TD (Nakahachi et al., 2006), and high-functioning autism (HFA) and TD groups (Losh et

al., 2009). Another study compared ASD, ADHD, and TD participants and showed that the ASD

group differed from both the ADHD and TD groups (Corbett et al., 2009). However, each of the

aforementioned mentioned studies used either a different version of the trails task (DKEFS or

ATMT) or a different task type (switching vs. sequencing.) These slight differences in effect,

make it harder to compare results across studies as they may be measuring similar, but slightly

different parts of EF. Including the time to completion as well as different types of errors

(sequencing or set-shifting) provides a more comprehensive picture of EF deficits.

1.4.6 Tower Test: Cognitive flexibility, task control

The D-KEFS tower test (Delis et al., 2001) is a table task that requires a participant move disks

of varying sizes across three pegs in order to match their tower to the specified picture using the

fewest possible number of moves. When administered correctly, the complexity of the task

enables the measurement of multiple EF domains including cognitive flexibility and task control.

The tower task was video and audio recorded, and then reviewed by both the initial tester

themselves and a secondary rater to verify the accuracy of both administration and scoring.

Variables included the mean accuracy of the time to first move, overall accuracy, rule violations

per item administered, time per move, total achievement score, and total rule violations. All of

the included variables were either scaled scores or percentiles.

7

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

Researchers using a similarly constructed task showed no significant differences between ASD

and TD individuals on a MANCOVA for Planning (minimum moves, initial thinking, and

subsequent thinking) (Corbett et al., 2009). HFA participants have also shown associations

between planning deficits on the tower task and reduced efficiency in visuospatial short-term

memory (Zinke et al., 2010). Another group showed that ASD youth had the lowest global

performance compared to ADHD and TD groups, but that those with comorbid ASD and ADHD

showed greater improvement trajectories than those that were ASD alone (Unterrainer et al.,

2015). Inclusion of all of the tower variables might enable for a better understanding of how

these impairments relate to other cognitive domains in individuals with ASD and/or ADHD.

1.4.7 Stop Task: Response inhibition, task control

The go/no-go Stop Task (Logan, 1994; Nigg, 1999) measures a participant’s ability to both react

quickly to a stimuli and to inhibit a response. In brief, participants fixate on a white screen and

are presented with a rainbow “X” or “O.” They are asked to either make a key press

corresponding to the X or O, or to inhibit their response at the presentation of an auditory tone.

The variables used in the model included an accuracy measure of X/O key press on go-trials, a

probability measure of inhibition on stop-trials, the stop signal reaction time, mean reaction time

on go-trials, and the standard deviation of reaction times on go-trials. While accuracy of X/O key

press on go-trials serves as a control for other task variables, it may also provide information

about letter anticipation which may be task-control related and thus was included in the models.

All justifications for including data in the FRF should be carefully considered, but another

benefit of the FRF is that it successfully ignores features that show no valuable contribution.

Because the stop task is not scaled, potential age confounds were also examined in the

supplementary analyses.

8

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

It has been widely demonstrated that participants with ADHD perform worse than their TD peers

on the stop task (Senderecka et al., 2012). Yet for one study that covaried for ADHD symptoms

in ASD and ADHD groups, these differences all but disappeared – with the exception of the

ASD group showing increased premature responses (Carter Leno et al., 2017). Meta-analyses on

response inhibition in ASD, including the stop-task, further emphasized that heterogeneity of

ASD may have an effect on the inconsistent results (Hilde M. Geurts et al., 2014). As evidenced

by the inconsistencies across the literature. Similar to other researchers, consideration of

comorbidities such as ADHD is recommended, as well as looking further into ASD subtypes.

1.4.8 Spatial Span: Working memory

The spatial span (Robbins et al., 1994)is a computerized task that measures spatial working

memory. The task presents 10 white boxes in random locations on the screen – a subset of which

change color, one at a time, in a fixed order. In the Forward task, upon completion of the color

change sequence, subjects hear a tone and are asked to click on the boxes in the order in which

they changed color on the screen. The number of squares that change color range from 3 to 9,

with two trials for each span length, and the task discontinues when a child fails both trials of the

same span. The Backward task is presented in the same way, but instead, subjects are asked to

click on the boxes in the opposite order in which they appeared. The forward and backward

spans were counterbalanced and subjects had the opportunity to practice prior to administration

of the task. Because the spatial span task is not scaled, potential age confounds were further

analyzed.

Several recent studies have suggested both impaired (Chen et al., 2016) and non-impaired

(Macizo et al., 2016) spatial working memory among those with ASD. Stronger evidence exists

to support such an impairment among those with ADHD, as they have been shown to exhibit

9

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

deficits in visio-spatial working memory as compared to TD peers across multiple studies

(Kasper et al., 2012). A study comparing participants with ADHD, ASD+ADHD, ASD, and TD

on a spatial working memory task showed that ADHD participants performed worse than both

the TD and ASD+ADHD groups and that both ASD+ADHD and ADHD groups needed longer

to perform the task than TD and ASD (Sinzig et al., 2008).

Response Inhibition Variable importance per model

Measure Variable Type Hyperactive InattentiveBRIEF Emotional Control T-score 0.1928 0.04579

BRIEF Inhibit T-score 0.855 -0.0039

BRIEF Monitor T-score 0.0802 0.0524

Color Word Total errors Scaled -0.0926 0.038

Color Word Total time Scaled -0.001 -0.0656

Color Word Total self-corrected errors Scaled -0.0184 0.0226

Color Word Total Uncorrected errors Scaled -0.0395 -0.0294

Stop Task Mean reaction time Raw 0.0942 -0.0418

Stop Task Standard Deviation on Go trials Raw 0.0959 0.0368

Stop Task Stop stimulus reaction time average Raw 0.053 0.0693

Stop Task Probability of stopping Raw 0.0094 -0.0188

Table 1: Eleven EF variables representing response inhibition included in the FRF models. On the right, the variable importance is shown per model.

Working Memory Variable importance per model

Measure Variable Type Hyperactive InattentiveBRIEF Working Memory T-score 0.0297 0.5102

Digit Span Longest digit recalled backward Percentile 0.0153 -0.0261

Digit Span Longest digit recalled forward Scaled 0.0101 0.0064

Digit Span Total backward score Scaled 0.0291 -0.0416

Digit Span Total forward score Scaled -0.0379 -0.0427

Digit Span Total, forward and backward Scaled 0.0083 -0.0238

Spatial Span Backward, mean reaction time Raw 0.2225 0.0888

Spatial Span Backward, Items correct (total accuracy) Raw 0.1634 0.1141

Spatial Span Backward, items correct/items attempted Raw 0.1359 0.0785

Spatial Span Forward, mean reaction time Raw -0.0124 0.0617

Spatial Span Forward, Items correct (total accuracy) Raw -0.0323 -0.0197

Spatial Span Forward, items correct/items attempted Raw 0.0387 -0.0161

Table 2: Twelve EF variables representing working memory included in the FRF models, with variable importance per model on the right.

10

210

211

212

213

214

215216

217

218219

220

Table 3: Ten EF variables representing cognitive flexibility included in the FRF models, with variable importance per model on the right.

Task Control Variable importance per model

Measure Variable Type Hyperactive InattentiveBRIEF Initiate T-score 0.112 0.3668

BRIEF Org. of materials T-score 0.0143 0.1556

BRIEF Plan and organize T-score 0.1253 0.4186

Tower Accuracy Scaled 0.0259 0.0204

Tower Rule violations per item Scaled -0.0022 -0.0106

Tower Total achievement Scaled 0.0236 0.0231

Tower Total rule violations Percentile -0.0136 -0.0099

Trails Sequencing errors Percentile 0.0765 -0.0485

Trails Total errors Scaled -0.0081 -0.0099

Stop Task Accuracy on Go trials Raw 0.1799 0.1292

Table 4: Ten EF variables representing task control included in the FRF models, with variable importance per model on the right.

1.5 fMRI Data

1.5.1 fMRI data acquisition

Participants were scanned at OHSU’s Advanced Imaging Research Center (AIRC) on a 3.0 T

Siemens Tim Trio Magnetom scanner using a 12 channel head coil. One T1 weighted structural

image (TR = 2300 ms, TE = 3.58 ms, orientation = sagittal, FOV = 256 × 256 matrix, voxel

resolution = 1mmx1mmx1.1 mm slice thickness) was acquired for each participant. fMRI data

(TR=2500ms, TE=30ms, flip angle=90°, voxel resolution = 3.75mm x 3.75mm x 3.80mm) were

collected as three, 5-minute BOLD resting state scans during which the participants were asked

to lie still and look at a white fixation cross on a black display.

11

Cognitive Flexibility Variable importance per model

Measure Variable Type Hyperactive InattentiveBRIEF Behavior regulation index T-score 0.3786 0.0622

BRIEF Metacognition T-score 0.147 0.4029

BRIEF Shift T-score -0.0697 0.1285

Tower Mean 1st move time Scaled 0.109 -0.0355

Tower Time per move Scaled 0.0968 -0.018

Trails Set loss errors Percentile 0.0482 -0.045

Trails Time to completion Scaled 0.004 0.0315

Verbal Fluency Contrast score Scaled -0.0409 -0.0368

Verbal Fluency WGI (Phonemic) Scaled 0.0257 -0.0537

Verbal Fluency WGS (Semantic) Scaled 0.0057 0.013

221222223224225226

227

228229

230

231

232

233

234

235

236

237

238

1.5.2 fMRI data processing

The HCP pipeline consists of five stages: 1) PreFreeSurfer, which corrects for MR gradient and

bias field distortions, aligns T1 images to a template, and registers the structural data to MNI

space; 2) FreeSurfer, which performs segmentation of brain volumes into predetermined regions,

reconstructs the surfaces, and aligns the images to a standardized template; 3) PostFreeSurfer,

which converts the data into standard NIH formats (Li et al., 2016), and applies the surface

registration to a template, and creates a final “brain mask;” 4) fMRIVolume, which removes

spatial distortions, performs motion correction, aligns the fMRI data to the participant’s

anatomical data, normalizes data to a global mean, and masks the data with the final brain mask;

and 5) fMRISurface, which maps the time series to a standard CIFTI format – which represents

the brain in grayordinate space (Glasser et al., 2013). Additional functional connectivity

processing was performed in alignment with current neuroimaging best practices (Mills et al.,

2018; Power et al., 2013). The additional steps included removing spikes in the MR signal, slice

time corrections, accounting and correcting for head motion during and between runs,

normalizing intensity to a whole brain value of 1000, temporal band-pass filtering, and

regressing out nuisance variables of motion estimates, tissue-based signals, and global signal

regression.

1.5.3 Quality control

Prior to performing manual quality control (QC) on data used in analyses, raters completed an

hour long training session followed by scoring a 10 participant reliability set that required an

inter-rater (anchor vs. rater) agreement of .8 to pass reliability. To score the agreement, half-

credit was given for disagreements between a high (1) and medium (2) quality scan, and no

credit was given for a misidentified low quality scan (3). Reliability training and manual QC of

12

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

data for the analyses worked in three steps. In the first step, T1 registration to the brain atlas were

visually assessed and determined to be of high (1), medium (2), or low (3) quality, with a high

quality image showing good registration and little to no warping. The second step involved a

visual inspection of the structural T1 images. A high quality T1 image shows well delineated

gray and white matter whereas a low quality image might show excessive blurring, ringing, or

sawtooth jaggedness in the delineation. Images deemed to be of high or medium quality were

then assessed for good functional registration using the same three tiered metric. High quality

functional registration shows that the data are well mapped to the T1 image across all

dimensions. A poor quality image might show things such as movement artifacts, field of view

cutoffs, distortions, or non-homologous signal or signal dropout. For the analyses, 1’s were

approved, 3’s were rejected, and scans that were a 2 in the functional and/or anatomical stage

were then assessed by a second rater to increase reliability and decrease bias before being

considered for inclusion. We further accounted for the effect of head movement on the MR

signal using motion-targeted “scrubbing”; censoring volumes with framewise displacement

greater than .2mm and eliminating any runs with less than 60 volumes total and/or less than 5

minutes of good resting state data. Only scans that passed all of the above criteria were

considered suitable for analysis.

2. Supplemental Analyses: Methods

13

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

2.1 The effect of age

2.1.1 Examining age differences between ASD and ADHD groups and the effect on the

models

In the main manuscript, we describe our cohort consisting of both ASD and ADHD participants.

Although we were not interested in comparing the ASD and ADHD groups to one another, the

possibility of a batch effect required that we investigate potential group differences. For the first

comparison between the ASD and ADHD participants, we identified a significant age difference

(t(128) = 3.67, p <.001.) We conducted several additional analysis to further examine the effects

of age on the models.

If age were a driving factor, we would expect to see a statistically significant increase in the

accuracy of the models when participant age is included as a 44th variable. Therefore, we ran

additional FRF models including age as an added variable for both the hyperactive and

inattentive models. We then compared the accuracy of the old models to the new models with t-

tests.

2.1.2 Age and unscaled EF variables

Nearly all of the variables were scaled, percentile, or t-scores. However, a handful of variables

were represented as raw scores (see Tables 1, 2, 3, 4.) Because age may be associated with task

performance, we ran a simple correlation between all raw EF measures and age to identify

potential relationships. We then examined variable importance per model to see if any of the

variables identified as highly correlated with age were also frequently used in the models, as any

variables that met both criteria could be considered potential confounds.

2.2 Differences between those with and without usable scan data

Out of the 130 participants, 67 were found to have useable scan data after quality control. We

14

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

wanted to determine if the 67 with useable scan data showed similar demographic profile

compositions to those without imaging data (n = 64). To do this, we ran significance tests

comparing demographic profiles for those without and with usable scan data. This included

diagnosis of ASD vs. ADHD (Fishers exact), gender (Fishers exact), age (t-test), and estimated

IQ (t-test.)

As in the main manuscript, we also wanted to ensure that those with good imaging data in the

identified subgroups (HSG-Mild = 38, HSG-Severe = 29; ISG-Mild = 20, ISG-Severe = 47) did

not represent significant demographic differences between subgroups per model. To do this, we

ran the same significance tests comparing demographic profiles for only those with good

imaging data in identified subgroups for both of the models.

We were also interested to see if those with good imaging data in the identified subgroups

showed similar neurocognitive profiles to the larger subgroups. Therefore, we ran additional t-

tests on the EF variables between subgroup participants with good imaging data and compared

them to the neurocognitive profiles in the main manuscript.

3. Results

3.1 The effects of age

3.1.1 Model accuracy and age as an additional variable

For both the hyperactive and inattentive models, t-tests on accuracy comparing the models with

and without age included as a variable showed no statistically significant difference in the model

performances (Hyperactive: t(58) = .501, p = .61, Inattentive: t(58) = -.14, p = .89.)

3.1.2 Age and EF scores

15

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

Significant results for the simple correlation between age and raw EF scores, corrected for

multiple comparisons, were observed on some of the measures (Table 5). When we examined

variable importance (Tables 1, 2, 3, 4), we found that none of the variables that were correlated

with age were important for the models.

Age correlated to raw EF tasks

Measure Variable Correlation

Stop Task Standard Deviation on Go trials r(127) -.340, p < .001

Stop Task Stop stimulus reaction time average r(121) -.340, p < .001

Spatial Span Backward, items correct (total accuracy) r(125) -.426, p < .001

Spatial Span Backward, mean reaction time r(125) .383, p < .001

Spatial Span Backward, items correct/items attempted r(125) .388, p < .001

Spatial Span Forward, items correct (total accuracy) r(124) -.355, p < .001

Spatial Span Forward, mean reaction time r(124) .333, p < .001Table 5: Significant correlations between age and raw EF tasks are shown in the table with the measure on the left, specific variable in the center column, and the correlation statistic on the right. Degrees of freedom are in the parentheses.

3.2 Demographic comparisons

3.2.1 Demographic profiles for those with and without useable scan data

Significance tests on demographic information (age, gender, IQ, and diagnostic composition)

comparing those with and without useable scan data can be seen in Table 6. Both primary

diagnostic assignment (ASD or ADHD) and age were significantly different between those with

and without useable scan data.

Unusable/no scan data (n = 64) Useable scan data (n = 67) p-valuea

ASD, 38 (59.4%) 26 (38.8%) p = .022

Female gender, n = 16 (25.4%) 15 (22.4) p = .84

Age (7-16y), m = 10.8 (1.8) 12.2 (2.3) t(124.4) = -3.78, p <.001

Est. IQ (Block design) m = 11.1 (3.25) 10.8 (3.2) t(65) = .615, p = .54

Table 6: Demographic comparisons for those with/without usable scan data

3.2.2 Demographic comparisons between identified subgroups for those with good scan

data

16

330

331

332

333

334335336

337

338

339

340

341

342

343

344

345

346

The hyperactive subgroups with good scan data did not show significant differences from each

other on age, gender, IQ, or diagnostic composition (Table 7).

Similarly, the inattentive subgroups with good scan data did not show significant differences

from each other on age, gender, IQ, or diagnostic composition (Table 8).

Useable scan data (n = 67) HSG-Mild (n = 38) HSG-Severe (n = 29) Significance testa

ASD, 26 (39%) 16 (42.1) 10 (34.5) p = .616

Female gender, 15 (22.4%) 7 (18.4) 8 (27.6) p = .393

Age in years (7-16y) m = 12.2 (2.3) 12.4 (2) 11.9 (2.6) t(50.75) = .79, p = .431

Est. IQ (Block design) m = 10.8 (3.2) 10.6 (3.4) 11.1 (2.8) t(65) = -.624, p = .524

Table 7: Demographic comparisons for those with imaging data in the Hyperactive subgroups.

Useable scan data (n = 67) ISG-Mild (n = 20) ISG-Severe (n = 47) Significance testa

ASD, 26 (39%) 9 (45) 17 (36) p = .587

Female gender, 15 (22.4%) 5 (25) 10 (21.3) p = .756

Age in years (7-16y) m = 12.2 (2.3) 12 (2.5) 12.2 (2.2) t(65) = -.301, p = .764

Est. IQ (Block design) m = 10.8 (3.2) 10.6 (2.9) 10.9 (3.3) t(65) = -.4, p = .688

Table 8: Demographic comparisons for those with imaging data in the Inattentive subgroups.

3.3 Neurocognitive profiles for those with useable scan data

The HSG-Mild and HSG-Severe groups mirrored the main manuscript in that the HSG-Mild

subgroup out-performed or had better ratings than the HSG-Severe group across several EF

measures (Figure 1a and Table 9). Four of the five measures were also found to be significantly

different between the HSGs in the main manuscript analysis (BRIEF: Behavioral regulation,

emotional control, inhibit, and Stop Task: go-trial accuracy.) An additional variable, Trails: Total

errors, was also found to be significantly different between the groups.

The ISG-Mild and ISG-Severe groups also showed similarities to the main manuscript analysis

in that the Mild subgroup out-performed or had better ratings than the Severe group across

several EF measures (Figure 1b and Table 10). Six of the eight measures were also observed in

the main manuscript analysis (BRIEF: Initiate, metacognition, monitor, organization of

17

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

materials, plan and organize, working memory.) The ISG-Mild group committed more rule

violations on the Tower of Hanoi, both per item and overall, than the ISG-Severe group.

Figure 1a: Behavioral measures used in the FRF Hyperactive model that significantly differed between subjects with imaging data in the identified subgroups. Normed means (y axis) from Table 9 are plotted per measure on the x axis in a line plot. To better represent the true differences when compared to a normative sample, all measures have been normed to the TD group.1b: Behavioral measures used in the FRF Inattentive model that significantly differed between subjects with imaging data in the identified subgroups. Normed means (y axis) from Table 10 are plotted per measure on the x axis in a line plot. Again, all measures have been normed to the TD group.

Table 9: Normed means compared between HSG's with good imaging data, and plotted in Figure 1a.

Response Inhibition t-test ISG-Severe ISG-Mild

BRIEF: Monitor t(24) = -3.67, p<.001 -2.56 -1.09

Cognitive Flexibility      

BRIEF: Metacognition t(65) = -10.25, p<.001 -2.95 -0.90

Working Memory      

18

Response Inhibition t-test HSG-Severe HSG-Mild

BRIEF: Inhibit t(59.9) = 11.43, p<.001 -4.55 -1.4

BRIEF: Emotional control t(65) = 3.46, p <.001 -2.85 -1.49

Stop task: Accuracy on Go-trials t(45.1) = 2.15, p = .037 -1.26 -0.33

Cognitive Flexibility      

BRIEF: Behavior regulation index t(65) = 6.31, p<.001 -3.95 -1.82

Task Control      

Trails: Total errors t(36.5) = 2.16, p = .037 -1.37 0.10

365

366

367

368369370371372373374

375376377378379

380

382383384385386387

388389390

BRIEF: Working memory t(65) = -7.12, p<.001 -3.25 -1.23

Task Control      

BRIEF: Initiate t(65) = -7.88, p<.001 -1.67 -0.33

BRIEF: Plan and organize t(31.1) = -8.83, p<.001 -2.72 -0.78

Tower: Rule viol. Percentile t(25.2) = -2.09, p<.001 -1.04 -3.22

Tower: Rule viol. p/item t(128) = -9.70, p<.001 -0.76 -2.05Table 10: Normed means compared between ISG's with good imaging data, and plotted in Figure 1b.

4. Discussion

While it cannot be entirely ruled out, our supplemental analyses suggest that age may not have

impacted model performance. Future studies should carefully consider group matching or other

methods to account for any age discrepancies.

In-scanner motion is a considerable issue among adolescent neuroimaging (Satterthwaite et al.,

2012). It remains a particular problem among pediatric and neurodivergent populations, wherein

“holding still” for prolonged periods of time may be both a function of age as well as a primary

deficit of the disorder/s. Researchers often observer that older participants tend to have better

methods for withstanding lengthy neuroimaging sessions. Therefore, in regards to our study, it is

somewhat expected that those with useable scan data are significantly older than those without.

While we believe we have achieved noteable success in managing participant motion through

mock –MRI sessions and appropriate behavioral interventions, additional techniques to control

for motion should be considered in future studies.

Along with comorbid ADHD features, those with ASD bear the additional burden of sensory

sensitivities, making achieving quality scan data a challenge. We observed a significant

difference in the number of participants with useable scan data when comparing ASD and

ADHD groups, with the number of ASD participants with useable scan data (26/64, 40%)

significantly less than expected as compared to those with ADHD and useable scan data (41/66,

19

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

62%.) Again, we were not comparing groups and used a transdiagnostic model to identify

subgroups. Although we “lost” more ASD participants in the imaging component, we did not

observe any significant demographic differences between both the identified subgroups for the

overall models (see: main manuscript) as well as when comparing subgroups for only those with

good imaging data (Tables 9 and 10).

As previously mentioned, those with greater ADHD and ASD symptoms may subsequently

struggle with things like holding still and withstanding scan duration. Considering only those

with useable neuroimaging data in the overall behavioral models would create a complication by

potentially eliminating those that may show the greatest impairments in EF and ADHD

symptoms. Therefore, we felt it important to consider as many subjects as possible in the main

models to best capture impairments and the relationships between EF and ADHD symptoms.

5. References

Adams, N. C., & Jarrold, C. (2012). Inhibition in autism: Children with autism have difficulty inhibiting irrelevant distractors but not prepotent responses. Journal of Autism and Developmental Disorders, 42(6), 1052–1063. https://doi.org/10.1007/s10803-011-1345-3

Begeer, S., Wierda, M., Scheeren, A. M., Teunisse, J. P., Koot, H. M., & Geurts, H. M. (2014). Verbal fluency in children with autism spectrum disorders: Clustering and switching strategies. Autism, 18(8), 1014–1018. https://doi.org/10.1177/1362361313500381

Bennetto, L., Pennington, B. F., & Rogers, S. J. (1996). Intact and impaired memory functions in autism. Child Development, 67(4), 1816–1835. https://doi.org/10.1111/j.1467-8624.1996.tb01830.x

Bishop, D. (2013). Children’s Communication Checklist (CCC-2). In Encyclopedia of Autism Spectrum Disorders (pp. 614–618). https://doi.org/10.1007/978-1-4419-1698-3_1929

Carter Leno, V., Chandler, S., White, P., Pickles, A., Baird, G., Hobson, C., Smith, A. B., Charman, T., Rubia, K., & Simonoff, E. (2017). Testing the specificity of executive functioning impairments in adolescents with ADHD, ODD/CD and ASD. European Child and Adolescent Psychiatry, 1–10. https://doi.org/10.1007/s00787-017-1089-5

Chen, S.-F. F., Chien, Y.-L. L., Wu, C.-T. T., Shang, C.-Y. Y., Wu, Y.-Y. Y., & Gau, S. S. (2016). Deficits in executive functions among youths with autism spectrum disorders: an age-stratified analysis. Psychological Medicine, 46(8), 1625–1638. https://doi.org/10.1017/S0033291715002238

20

411

412

413

414

415

416

417

418

419

420

421

422

423

424425426427428429430431432433434435436437438439440441442443

Conners, C. K., Pitkanen, J., & Rzepa, S. R. (2011). Conners 3rd Edition (Conners 3; Conners 2008). In Encyclopedia of Clinical Neuropsychology (pp. 675–678). https://doi.org/10.1007/978-0-387-79948-3_1534

Constantino, J., & Gruber, C. (2005). Social responsiveness scale. Western Psychological Services.

Corbett, B. A., Constantine, L. J., Hendren, R., Rocke, D., & Ozonoff, S. (2009). Examining executive functioning in children with autism spectrum disorder, attention deficit hyperactivity disorder and typical development. Psychiatry Research, 166(2–3), 210–222. https://doi.org/S0165-1781(08)00052-8 [pii] 10.1016/j.psychres.2008.02.005

Delis, D. C., Kaplan, E., & Kramer, J. (2001). Delis Kaplan Executive Function System. The Psychological Corporation.

DuPaul, G. J., Reid, R., Anastopoulos, A. D., Lambert, M. C., Watkins, M. W., & Power, T. J. (2016). Parent and teacher ratings of attention-deficit/hyperactivity disorder symptoms: Factor structure and normative data. Psychological Assessment. https://doi.org/10.1037/pas0000166

Gardiner, E., & Iarocci, G. (2017). Everyday executive function predicts adaptive and internalizing behavior among children with and without autism spectrum disorder. Autism Research. https://doi.org/10.1002/aur.1877

Geurts, H M, Verte, S., Oosterlaan, J., Roeyers, H., & Sergeant, J. A. (2004). How specific are executive functioning deficits in attention deficit hyperactivity disorder and autism? Journal of Child Psychology and Psychiatry, 45(4), 836–854. https://doi.org/10.1111/j.1469-7610.2004.00276.xJCPP276 [pii]

Geurts, Hilde M., van den Bergh, S. F. W. M. W. M., & Ruzzano, L. (2014). Prepotent response inhibition and interference control in autism spectrum disorders: Two Meta-Analyses. Autism Research, 7(4), 407–420. https://doi.org/10.1002/aur.1369

Gioia, G. A., Isquith, P. K., Guy, S. C., Kenworthy, L., & Baron, I. S. (2001). Behavior rating inventory of executive function. In Child Neuropsychology. https://doi.org/10.1076/chin.6.3.235.3152

Glasser, M. F., Sotiropoulos, S. N., Wilson, J. A., Coalson, T. S., Fischl, B., Andersson, J. L., Xu, J., Jbabdi, S., Webster, M., Polimeni, J. R., Van Essen, D. C., & Jenkinson, M. (2013). The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage, 80, 105–124. https://doi.org/10.1016/j.neuroimage.2013.04.127

Goodman, R. (1997). The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry, 38(5), 581–586. https://doi.org/10.1111/j.1469-7610.1997.tb01545.x

Hurks, P. P. M., Hendriksen, J. G. M., Vles, J. S. H., Kalff, A. C., Feron, F. J. M., Kroes, M., Van Zeben, T. M. C. B., Steyaert, J., & Jolles, J. (2004). Verbal fluency over time as a measure of automatic and controlled processing in children with ADHD. Brain and Cognition. https://doi.org/10.1016/j.bandc.2004.03.003

Karalunas, S. L., Hawkey, E., Gustafsson, H., Miller, M., Langhorst, M., Cordova, M., Fair, D., & Nigg, J. T. (2018). Overlapping and Distinct Cognitive Impairments in Attention-Deficit/Hyperactivity and Autism Spectrum Disorder without Intellectual Disability. Journal of Abnormal Child Psychology. https://doi.org/10.1007/s10802-017-0394-2

Kasper, L. J., Alderson, R. M., & Hudec, K. L. (2012). Moderators of working memory deficits in children with attention-deficit/hyperactivity disorder (ADHD): A meta-analytic review. In Clinical Psychology Review. https://doi.org/10.1016/j.cpr.2012.07.001

Kaufman, J., Birmaher, B., Brent, D., Rao, U., Flynn, C., Moreci, P., Williamson, D., & Ryan,

21

444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489

N. (1997). Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry, 36(7), 980–988. https://doi.org/10.1097/00004583-199707000-00021

Kenworthy, L., Wallace, G. L., Birn, R., Milleville, S. C., Case, L. K., Bandettini, P. A., & Martin, A. (2013). Aberrant neural mediation of verbal fluency in autism spectrum disorders. Brain and Cognition, 83(2), 218–226. https://doi.org/10.1016/j.bandc.2013.08.003

Korkman, M., Kirk, U., & Kemp, S. (2007). NEPSY-Second Edition (NEPSY-II). Journal of Psychoeducational Assessment. https://doi.org/10.1177/0734282909346716

Kovacs, M. (1985). The Children’s Depression, Inventory (CDI). Psychopharmacology Bulletin, 21(4), 995–998. https://doi.org/10.3724/SP.J.1041.2015.01004

Li, X., Morgan, P. S., Ashburner, J., Smith, J., & Rorden, C. (2016). The first step for neuroimaging data analysis: DICOM to NIfTI conversion. Journal of Neuroscience Methods, 264, 47–56. https://doi.org/10.1016/j.jneumeth.2016.03.001

Logan, G. D. (1994). On the ability to inhibit thought and action: A users guide to the stop-signal paradigm. In Inhibitory processes in attention, memory, and language (pp. 189–239).

Lord, C, Rutter, M., DiLavore, P., Risi, S., & Gotham, K. (2012). Autism diagnostic observation schedule, (ADOS-2). Los Angeles, California.

Lord, Catherine, Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord, 24(5), 659–685. https://doi.org/10.1007/BF02172145

Losh, M., Adolphs, R., Poe, M. D., Couture, S., Penn, D., Baranek, G. T., & Piven, J. (2009). Neuropsychological profile of autism and the broad autism phenotype. Archives of General Psychiatry, 66(5), 518–526. https://doi.org/10.1001/archgenpsychiatry.2009.34

Lynch, C. J., Breeden, A. L., You, X., Ludlum, R., Gaillard, W. D., Kenworthy, L., & Vaidya, C. J. (2017). Executive Dysfunction in Autism Spectrum Disorder Is Associated With a Failure to Modulate Frontoparietal-insular Hub Architecture. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 2(6), 537–545. https://doi.org/10.1016/j.bpsc.2017.03.008

Macizo, P., Soriano, M. F., & Paredes, N. (2016). Phonological and Visuospatial Working Memory in Autism Spectrum Disorders. Journal of Autism and Developmental Disorders, 46(9), 2956–2967. https://doi.org/10.1007/s10803-016-2835-0

Mahone, E. M., Cirino, P. T., Cutting, L. E., Cerrone, P. M., Hagelthorn, K. M., Hiemenz, J. R., Singer, H. S., & Denckla, M. B. (2002). Validity of the behavior rating inventory of executive function in children with ADHD and/or Tourette syndrome. Arch Clin Neuropsychol, 17(7), 643–662. https://doi.org/S0887617701001688 [pii]

March, J. S. (2004). Multidimensional Anxiety Scale for Children (MASCTM). The Use of Psychological Testing for Treatment Planning and Outcomes Assessment: Instruments for Children and Adolescents, 39. https://doi.org/0805843302

Mills, B. D., Miranda-Dominguez, O., Mills, K. L., Earl, E., Cordova, M., Painter, J., Karalunas, S. L., Nigg, J. T., & Fair, D. A. (2018). ADHD and attentional control: Impaired segregation of task positive and task negative brain networks. Network Neuroscience (Cambridge, Mass.), 2(2), 200–217. https://doi.org/10.1162/netn_a_00034

Nakahachi, T., Iwase, M., Takahashi, H., Honaga, E., Sekiyama, R., Ukai, S., Ishii, R., Ishigami, W., Kajimoto, O., Yamashita, K., Hashimoto, R., Tanii, H., Shimizu, A., & Takeda, M.

22

490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535

(2006). Discrepancy of performance among working memory-related tasks in autism spectrum disorders was caused by task characteristics, apart from working memory, which could interfere with task execution. Psychiatry and Clinical Neurosciences, 60(3), 312–318. https://doi.org/10.1111/j.1440-1819.2006.01507.x

Nigg, J. T. (1999). The ADHD response-inhibition deficit as measured by the Stop Task: replication with DSM-IV combined type, extension, and qualification. J Abnorm Child Psychol, 27(5), 393–402. https://doi.org/10.1023/A:1021980002473

Nigg, J. T., Blaskey, L. G., Huang-Pollock, C. L., & Rappley, M. D. (2002). Neuropsychological executive functions and DSM-IV ADHD subtypes. J Am Acad Child Adolesc Psychiatry, 41(1), 59–66. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=11800208

Ozonoff, S., & Strayer, D. L. (2001). Further Evidence of Intact Working Memory in Autism. Journal of Autism and Developmental Disorders, 31(3), 257–263. https://doi.org/10.1023/A:1010794902139

Power, J., Mitra, A., Laumann, T., Snyder, A., Schlaggar, B., & Petersen, S. (2013). Methods to detect, characterize, and remove motion artifact in resting state fMRI. Neuroimage. https://doi.org/10.1016/j.neuroimage.2013.08.048

Robbins, T. W., James, M., Owen, A. M., Sahakian, B. J., McInnes, L., & Rabbitt, P. M. (1994). Cambridge Neuropsychological Test Automated Battery (CANTAB): a factor analytic study of a large sample of normal elderly volunteers. Dementia, 5(5), 266–281.

Satterthwaite, T., Wolf, D., Loughead, J., Ruparel, K., Elliott, M., Hakonarson, H., Gur, R., & Gur, R. (2012). Impact of in-scanner head motion on multiple measures of functional connectivity: relevance for studies of neurodevelopment in youth. Neuroimage, 60(1), 623–632. https://doi.org/10.1016/j.neuroimage.2011.12.063

Senderecka, M., Grabowska, A., Szewczyk, J., Gerc, K., & Chmylak, R. (2012). Response inhibition of children with ADHD in the stop-signal task: An event-related potential study. International Journal of Psychophysiology, 85(1), 93–105. https://doi.org/10.1016/j.ijpsycho.2011.05.007

Sinzig, J., Morsch, D., Bruning, N., Schmidt, M. H., & Lehmkuhl, G. (2008). Inhibition, flexibility, working memory and planning in autism spectrum disorders with and without comorbid ADHD-symptoms. Child and Adolescent Psychiatry and Mental Health, 2(1), 4. https://doi.org/10.1186/1753-2000-2-4

Spek, A., Schatorje, T., & Scholte, E. (2009). Verbal fluency in adults with high functioning autism or Asperger syndrome. Neuropsychologia, 47(3), 652–656. http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=emed9&NEWS=N&AN=19084028

Unterrainer, J. M., Rauh, R., Rahm, B., Hardt, J., Kaller, C. P., Klein, C., Paschke-Muller, M., Biscaldi, M., Paschke-M??ller, M., & Biscaldi, M. (2015). Development of Planning in Children with High-Functioning Autism Spectrum Disorders and/or Attention Deficit/Hyperactivity Disorder. Autism Res, 9(7), 739–751. https://doi.org/10.1002/aur.1574

Verté, S., Geurts, H. M., Roeyers, H., Oosterlaan, J., & Sergeant, J. A. (2006). Executive functioning in children with an autism spectrum disorder: Can we differentiate within the spectrum? Journal of Autism and Developmental Disorders, 36(3), 351–372. https://doi.org/10.1007/s10803-006-0074-5

Wechsler, D. (2003). Wechsler Intelligence Scale for Children—4th Edition (WISC-IV®). Harcourt Assessment.

23

536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581

Zinke, K., Fries, E., Altgassen, M., Kirschbaum, C., Dettenborn, L., & Kliegel, M. (2010). Visuospatial short-term memory explains deficits in tower task planning in high-functioning children with autism spectrum disorder. Child Neuropsychology, 16(3), 229–241. https://doi.org/10.1080/09297040903559648

24

582583584585586587