1

Good Progress but Challenges Remain: Achieving Equity in Fairfax County Public

Schools Advanced Academic Programs

Jonathan A. Plucker, Ph.D, Carolyn M. Callahan, Ph.D., Dante D. Dixson, Ph.D., and Scott J. Peters, Ph.D.

May 5, 2020

Reported Submitted in Fulfillment of FCPS RFP 3100000390 for Program Review Services for Advanced Academic Programs

2

Table of Contents Glossary ………………………………………………………………………………………………………………………………… 3 Description of Task and Supporting Data ………………………………………………………………………………. 5 Methodology ………………………………………………………………………………………………………………........... 10 Literature Review on Interventions ………………………………………………………………………………………. 14 Findings and Conclusions ………………………………………………………………………………………………………. 25 Recommendations ………………………………………………………………………………………………………………… 42 References ……………………………………………………………………………………………………………………………. 47 About the Review Team ……………………………………………………………………………………………………….. 52 Appendices …………………………………………………………………………………………………………………………… 55

A. Relevant sections of Virginia Code

B. Sample focus group and interview protocols

C. Parent survey

D. NNAT, CogAT, and GBRS score descriptive statistics for all students

E. NNAT, CogAT, and GBRS score descriptive statistics for Level IV-eligible students

F. Parent survey results summary

3

Glossary

Assessment bias: Differences in test scores or teacher ratings across groups do not by themselves indicate assessment bias. Such mean-score differences, or differences in Level IV identification rates across two groups, would only be considered evidence of bias if those differences are due to faulty measurement procedures and not due to real, underlying differences in what the tests are measuring. Broadly speaking, assessment bias occurs when there are differences between two parameters that should be equal. An unbiased system can still produce mean test score differences or differences in rates of gifted student identification across groups. Central Selection / Screening Committee: A group of FCPS educators who receive training about the holistic screening process and review student portfolios in order to make Level IV eligibility decisions. In 2019, this group met for the district-wide Level IV eligibility review in December and reviewed 6779 student files. Cognitive Abilities Test (CogAT): A widely-used measure of K-12 students’ reasoning abilities via quantitative, verbal, and nonverbal batteries. In FCPS, the CogAT is administered to all students in second grade. Disproportionality (or underrepresentation): The term disproportionality is commonly used to refer to the situation in which the proportion of students from a given subpopulation of students enrolled in advanced learning or gifted and talented programs does not mirror their proportion in the entire student population in terms of race, ethnicity, gender, home language, or disability status. Most commonly, it refers to cases where African American or Hispanic students are disproportionately underrepresented in AAP compared to their representation in the overall student population. Excellence Gaps: Gaps in advanced academic achievement or attainment across students of differing demographics. Free or Reduced Price Meals: This dichotomous variable is often used as a proxy for a family experiencing poverty or the stresses or lower family income. For participation in the National School Lunch Program, families whose combined income is 130% or less of the Federal poverty line are eligible for free meals at school. Gifted Behaviors Rating Scale (GBRS): An FCPS-specific measure used by teachers to rate the frequency of behaviors typical to gifted students / advanced learners. Teachers rate each of their students every year on the four GBRS items: exceptional ability to learn, exceptional application of knowledge, exceptional creative / productive thinking, and exceptional motivation to succeed. i-Ready Assessment: Diagnostic assessments in reading and mathematics that provide current achievement level data that can be used for instructional decision-making.

4

Level IV Pool: The group of students who are considered for Level IV services via the Central Selection Committee. This pool is made up of students who meet NNAT and CogAT criteria in grade 2 (2nd grade screening pool) or who were referred into the pool despite not having qualifying scores (referral candidates). Level IV Eligible vs. School Designated: Level IV eligible students are those who have been identified formally as Level IV students and offered spots in either a Level IV center or local Level IV program. Level IV school designated students are those who are identified as Level III students but participate in local Level IV programs if seats are available. Limited English Proficient Student (LEP – also known as English language learners or ELL): A student who is not yet proficient in the English language, typically because it is not his or her native language. Local Norms: A norm-referenced interpretation is any time a student’s test score is compared to the test scores of some other group (e.g., “Bobby is the best player on the team” compares Bobby to the rest of the team). “Local” norms differ from national norms in that the student’s score is compared to that of her grade-level peers in a particular school building (school norms) or school district (district norms). Naglieri Nonverbal Abilities Test (NNAT): A widely-used non-verbal measure of general ability with limited reliance on English (i.e., no written directions or items). In FCPS, the NNAT is administered to all students in first grade. National Norms: A norm-referenced interpretation is any time a student’s test score is compared to the test scores of some other group (e.g., “Bobby is the best player on the team” compares Bobby to the rest of the team). In the case of a national norm, the data reported is most often the percent of a national normative sample that the student performed better than – for example, a national percentile of 95 means the student scored better than 95% of national test takers (or a nationally-representative sample). Twice Exceptional: The term “twice exceptional,” also referred to as “2e,” is used to describe gifted children who have the characteristics of gifted students with the potential for high achievement and give evidence of one or more disabilities as defined by federal or state eligibility criteria (NAGC). Two-Phase Identification System: A process for identifying students for gifted programming (e.g., Level IV eligibility) whereby students first take a screening assessment(s) and only those who pass through the screening phase are put through the program eligibility process. FCPS uses a two-phase system for Level IV eligibility when all student take the NNAT in first-grade and CogAT in second grade. Only those students who score high enough on one or the other are considered by the Central Selection Committee for Level IV placement.

5

Achieving Greater Equity in FCPS Advanced Academic Programs The study was conducted in response to FCPS informal RFP 3100000390 for Program Review Services for Advanced Academic Programs. The focus of the proposed review was to combine an analysis of available FCPS data and documentation with data collected by the review team and state-of-the-art theory and research to create a set of recommendations for improving equity across all four levels of FCPS Advanced Academic Program (AAP) services based on the team’s recent research and direct experience on similar projects.1 We examined a broad range of data to determine the extent to which equity is an issue at various levels of AAP participation. We approached understanding the current demographic representation of AAP students in three ways:

1. Division-generated reports for enrollment in any/all AAP services, 2. Division-generated reports for Level IV eligibility over time, 3. Original analyses, based on student-level data of a single cohort of grade 2 students screened

for level IV services for the from 2018 – 2019 school year and comparing the demographics of the overall cohort to the sample screened for Level IV services and to the sample found eligible for Level IV services.

Division-generated Reports for Enrollment in AAP Services Demographic comparisons across Levels II – IV AAP services are difficult because each level (aside from Level IV Centers) involves 1) students being selected for participation based on varying criteria across different schools and/or 2) different service options offered across schools. For example, Local Level IV services are not available in each building and, even when they are available, schools may use different criteria for those who actually receive Local Level IV services. Similarly, principals and local school-level identification committees have more control over identification of students for placement of into Level II or Level III services. As a general statement, all racial/ethnic groups are represented in very similar proportions to their representation in the FCPS population in Level II and III services (see Tables 1-4 and Figure 1 below). For example, White, Asian, Hispanic, and Black students are represented in Level II services at rates of 13%, 13%, 11%, and 13% (see Table 2). However, the higher the level of service, the larger the racial disparities, particularly for Hispanic students. For instance (see Table 4), last year’s cohort of grade 2 students was made up of 9.8% Black students but they make up just 6.4% of those screened for Level IV services and only 7% of the cohort identified as Level IV eligible. In contrast, Hispanic students made up 25.3% of the full school cohort but just 10.6% of those screened and only 12.3% of those who were identified as Level IV eligible. This stark difference clearly demonstrates the identification gap is larger for Hispanic students than Black students. Notably, once a student is identified for one level of service, they represent a student who cannot be identified for any other level. For example, a Hispanic student identified for Level IV is now one less

1 At various points during the review team’s interaction with FCPS personnel and stakeholders, interpretations of the contracted task varied widely, including advice that we focus only on Level IV centers, that we address only Level IV local or center programming, or that we not address Level IV centers at all. However, the goal of the contract was clarified with FCPS leadership in December 2019, and throughout the project the review team collected and analyzed data and information related to improving equity at all four levels of AAP programming.

6

student who can be identified at Level III. This is the challenge at looking at demographic enrollment for each level of service. At the very least, it might also be useful to examine equity for AAP as a whole, compared to district enrollment, in addition to equity within each level of service. Further, it is important to emphasize that all of these data provide evidence regarding proportionality – the degree to which each level of service is or isn’t made up of the same demographic groups in the same proportions as the Division or cohort of students as a whole. Proportionality and eligibility or readiness are not the same thing, unless all groups have equal levels of readiness and need for a level of service – something we discuss at length below. Table 1. Participation Gaps in FCPS Advanced Academic Programs

Table 2. AAP Demographics by Level of Service – Fall 2019: Percent of each racial group in each level of service

TOTAL WHITE ASIAN BLACK HISPANIC OTHER

Total grades K-6 94820 36616 17925 8824 25137 6318

% of grades K-6 100% 38.6% 18.9% 9.3% 26.5% 6.6%

AAP Level IV (Determined by Central Selection Committee only)

10152 3998 3468 705 1126 855

10.7% 10.9% 19.3% 8.0% 4.5% 13.5%

AAP Level IV (Determined by Central Selection Committee and School Designated)

11209 4435 3788 768 1293 925

11.8% 12.1% 21.1% 8.7% 5.1% 14.6%

Level III services 6719 3288 1326 479 1091 535

7.1% 9.0% 7.4% 5.4% 4.3% 8.5%

Level II services 11672 4762 2237 1133 2733 807

12.3% 13.0% 12.5% 12.8% 10.9% 12.8%

Source: School AAP Enrollment File

7

Table 3. Students Eligible for Level IV (full-time grades 3-8) AAP Services Across the Years 2008-2018

Year White

% change

from prior year

Black

% change

from prior year

Hispanic

% change

from prior year

Asian

% change

from prior year

Two or More Races

% change

from prior year

Total

% change

from prior year

2008 5,360 382 403 2,821 703 9,687

2009 5,666 6% 400 5% 438 9% 3,209 14% 746 6% 10,475 8%

2010 5,461 -4% 351 -12% 580 32% 3,341 4% 614 -18% 10,361 -1%

2011 5,776 6% 415 18% 696 20% 3,676 10% 688 12% 11,271 9%

2012 6,763 17% 535 29% 899 29% 4,282 16% 835 21% 13,342 18%

2013 6,989 3% 611 14% 975 8% 4,625 8% 960 15% 14,192 6%

2014 7,089 1% 684 12% 1,142 17% 4,933 7% 1022 6% 14,913 5%

2015 7,062 0% 786 15% 1,231 8% 5,203 5% 1121 10% 15,446 4%

2016 7,174 2% 917 17% 1,452 18% 5,519 6% 1174 5% 16,282 5%

2017 7,294 2% 1,076 17% 1,689 16% 5,788 5% 1254 7% 17,153 5%

2018 6,975 -4% 1,092 1% 1,699 1% 5,728 -1% 1280 2% 16,829 -2%

% increase

2008-2018

30% 186% 322% 103% 82% 74%

Note. Subgroups with very small numbers of students removed for purposes of clarity. These include American Indian students (increase from 2008-2018 from 12 to 38 students, +217%) and Hawaiian students (increase from 6 to 17 students, +183%)

8

Demographic Breakdown of 2018-2019 Grade 2 Cohort As noted above, one of the data sources utilized in this report was the universal screening data set from the cohort of students who were enrolled in FCPS grade 2 for the 2018 – 2019 academic year. In FCPS, the Naglieri Nonverbal Ability Test (NNAT) is administered to all first-grade students, and the Cognitive Abilities Test (CogAT) is administered to all second-grade students. These data are used to create the initial screening “pool” – the students who are automatically considered for Level IV services via a Central Selection Committee. For the 2018 – 2019 school year, students needed to score a 132 (two standard deviations above the national mean score) on either the NNAT or composite CogAT in order to be placed in the automatic pool via the universal screening process (we discuss this process in greater detail below). Starting in 2019 – 2020, students could be placed in Level IV services if they scores two standard deviations above the mean on any one CogAT subscale. Our cohort dataset also included racial / ethnic information, gender, and scores each student received on the Gifted Behavior Rating Scale (GBRS) – an instrument completed by each students’ teacher on observable gifted characteristics. We used this cohort dataset to calculate racial / ethnic demographics for the cohort as a whole, the population of students placed in the screening pool, and the population of students who were identified as Level IV eligible. At least with regard to Level IV, this presentation is preferable for understanding racial / ethnic enrollment differences. These data are presented in Table 4. We also used this dataset to understand if certain student groups were less likely to be screened or identified when compared to similarly-scoring peers from other groups.

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018

Figure 1. Students Eligible for Level IV AAP,Grades 3-8, 2008-2017

White Black Hispanic American Indian Asian Two or More Races Hawaiian

9

Table 4. Demographic comparisons across groups – 2018/2019 Grade 2 Cohort (Level IV)

Percentage of Cohort1 Overall

Population of Students Screened (pool)

Percentage of Identified Cohort

African American 9.8% 6.4% 7%

Asian American 18.7% 30.7% 30.8%

European American 39% 43.2% 40%

Hispanic 25.3% 10.6% 12.3%

Multi-racial 6.7% 8.9% 9.6%

Total 100% (13818) 29.3% (3757) 16% (2198)

1“Cohort” refers to students in FCPS second-grade in 2018 – 2019 school year Table 4 is a clear illustration of the disproportional enrollment / equity challenges that are common to the AAP program in FCPS as well as gifted and talented programs across the country. Students from African American and Hispanic families are disproportionally underrepresented both in the screening pool as well as among those deemed Level IV eligible (or academically gifted in gifted and talented programs). In addition, Asian American students are disproportionately overrepresented in the “pool” and “identified” groups, while European American students are more-or-less proportionally represented across all three groups. These enrollment rate differences were a motivating factor behind the RFP and also the team’s investigation and recommendations.

10

Methodology This section provides an overview of the review team’s approach to data collection, organized by the four contracted tasks and additional parent survey: (1) a review of existing FCPS policies and related documents, (2) a review of existing FCPS data analyses and reports, (3) a review of research-based interventions and best practices, (4) focus groups and interviews with FCPS and community stakeholders, and (5) a survey of parent attitudes. Each set of activities is described in more detail below. Review of Policies and Related Documents The first data source included any and all policy / procedure documents, school board presentations, and internal analyses produced by FCPS related to AAP. These were reviewed by the entire research team through the lens of best practices for gifted and talented student identification and procedures known to exacerbate or mitigate inequity. Many of the findings from these past data sources are referenced in this report. This includes a number of tables and figures showing historical demographic trends. Some of the most important FCPS documents or analyses reviewed include:

1. FCPS Local Plan for the Education of the Gifted 2016-2021 (https://www.fcps.edu/sites/default/files/media/pdf/LocalPlanGifted2016to2021.pdf)

2. FCPS Advanced Academic Programs Identification Procedures Manual: 2019-2020 (see AAP web site for this and related documents: https://www.fcps.edu/academics/academic-overview/advanced-academic-programs)

3. 2013 Review of the Advanced Academic Programs authored by Bland et al. 4. May 2018 and May 2019 Advanced Academic Programs Advisory Committee Reports and Staff

Responses to Recommendations 5. Training materials for Level IV Central Selection Committee members 6. 2019 Minority Student Achievement Oversight Committee (MSAOC) History of Reports to the

FCPS School Board 1990 – Present (provided by J. Howard) 7. Data on historical pool benchmark scores for NNAT and CogAT 8. Expansive report on hypothetical alternative identification criteria (e.g., different benchmark

scores, local norms) 9. 2010 – 2019 demographic breakdown of students placed into consideration pool by test scores 10. 2010 – 2019 demographic breakdown of students placed into the consideration pool by referrals 11. Referral data showing the source of AAP referrals by race, ethnicity, gender, ELP, and Young

Scholar participation for 2016 – 2019 school year (called “cohort” above) 12. GBRS instrument and associated training information for teachers 13. Chapter 40: Regulations Governing Educational Services for Gifted Students (§ 22.1-16 of the

Code of Virginia; relevant sections included in the Appendix to this report) Review of Existing Data and Analyses In addition to relying on and reviewing existing reports, we also requested and analyzed data from FCPS staff. In some cases, the, FCPS staff conducted analyses; at our request, we were provided with the raw data used to conduct our own analyses.

11

1. Primary analyses a. iReady data on average student achievement by grade level and building b. Universal screening data (GBRS, CogAT, NNAT) for all grade 2 students in 2018 – 2019

cohort. We used complete data from the 2018 – 2019 grade 2 cohort in order to conduct a number of analyses on the identification process. Focusing on a single year / cohort in this way allowed us to evaluate the formal identification process – in short, to examine leaks in the pipeline. Analyses included:

i. Comparing mean scores by test by student subgroup ii. Comparing % of each student subgroup in the overall cohort population,

screened sample, and sample found eligible for Level IV services (Table 4 above) iii. Measuring the odds of a student being found Level IV eligible by gender and

race / ethnicity after controlling for NNAT, CogAT, and GBRS scores 2. Requested analyses from FCPS

Data on % of students served for each level of FCPS AAP by race, ethnicity, IDEA, and LEP

Data on number and percentage of students screened by attending school

Choice of placement for Level IV eligible students for the past three academic years: Center, local Level IV, or deferred.

Level IV Center discontinuation rate / placement in year following Level IV Center placement disaggregated by race, ethnicity, IDEA, ELP, and 504 status

Data on eligibility decision following an appeal disaggregated by race / ethnicity

Data on rates and source of outside testing by race / ethnicity

Data on rates of referral (and referral source) over time

AAP participation data by level of service and race / ethnicity Review of Research-based Interventions and Best Practices The team conducted a comprehensive review of the literature to identify empirically-supported identification and programming interventions for increasing equity in advanced programming. Given that book-length treatments of these topics are not uncommon (indeed, review team members have written or contributed to several such volumes), the goal for this review was to highlight key best practices for achieving equity in advanced programs in the research literature. Focus Groups and Interviews with Stakeholders The first step in the process of planning for focus groups and individual interviews was to establish the stakeholder groups most likely to provide insights into the current state of affairs around issues of equity in the AAP program. The four reviewers prepared a list of groups of individuals to be interviewed by role (e.g., parents, AAP teachers, Advanced Academic Resource Teachers (AARTs)) and by committee assignments within FCPS, representing varying orientations, interests, and perspectives on the issues (e.g., concerns from members of historically under-represented populations (in gifted programs) or marginalized groups, twice-exceptional student involvement, identification of gifted students). We then met with staff from FCPS representing the AAP program, the central administration, and a liaison from the Office of Research and Strategic Improvement (ORSI) to (1) ensure our list was

12

comprehensive, (2) develop a plan for identifying specific individuals from each group, (3) develop a plan for communicating with the stakeholders to be invited to participate, and (4) ensure compliance with guidelines for carrying out such reviews as established by FCPS. The discussion also included an exploration of ways to ensure representativeness of all groups, development of a viable schedule, and development of a plan to ensure travel by participants would be manageable and not bias attendance. We determined that situating interviewers at schools within the different regions across the school division would minimize travel for school-based participants (e.g., parents, teachers, principals) in focus groups and interviews, and that other interviews and focus groups would be scheduled at the central office. Following the meeting with FCPS staff, we created a spreadsheet to select a sample of representative schools. We randomly selected elementary schools based on the following criteria:

Group A: Schools in Regions 1 and 2 with high minority representation, a high percentage of students that receive free/reduced-price meals (FRM), and representatives across all levels of AAP of programming. At least two schools selected would offer the Young Scholars program.

Group B: Schools in Regions 3 and 4 with relatively high minority representation, a high percent of students receiving FRM, and representatives across all levels of AAP programming

Group C: Schools from Regions 4 and 5 –mostly 5—with relatively low minority representation, relatively low percent of students receiving FRM, and representatives across all levels of AAP of programming

Group D: Schools from Regions 1 and 2 –mixed across all variables noted above

Group E: Schools from Regions 3, 4, and 5- mixed across all variables noted above

Middle Schools were selected to represent all five regions with the inclusion of the one Title I school. The liaison from ORSI provided school demographic information from which the review team selected schools from each region. ORSI also assisted with random sampling of teacher, parent, and student names that were used for focus groups. All principals from schools selected in each region were invited to attend the focus group at the school selected to be the host school in the region; a random selection of teachers, AARTs, students identified for AAP services, parents of students identified for AAP, and members of the local AAP identification committee from each host school were invited to attend focus group sessions. Interview Guides. In preparing for the focus group meetings and individual interviews, we created a series of tailored interview guides (see Appendix). These interview guides were structured to provide a general framework for questioning. Each guide was comprised of questions to be asked during the meetings, and the questions were created to reflect issues of equity in the AAP program and to provide a general sense of the orientation and views of the participants toward the program’s identification process and service delivery models as they might relate to equity. The guides were prepared to ensure that the same basic lines of inquiry were pursued with each group or individual. However, additional questions were added to each guide that would allow us to capitalize on the knowledge and understandings that a person or group might offer because of their role or experiences with the school division and/or the AAP program. The guides were used as a framework; however, the interviewer was free to explore, probe, and ask questions that elucidated or clarified responses or would provide verification of responses from other people or groups interviewed. Thus, the people who were interviewing, on occasion, might build a conversation around a particular subject area or word questions spontaneously or even pursue a new line of questioning if responses suggested the line of questioning would yield relevant information for the review.

13

Data Analysis. During the focus group and individual interviews, the interviewers took notes on participants’ responses, particularly focusing on responses that provided insight into equity issues. Following each visit, the interviewers wrote summary notes of the responses from interviews and focus groups. These notes were shared with the whole team. The team reviewed the notes and discussed them with a particular emphasis on comments and observations that suggested a need for pursuit of any emerging themes at a subsequent visit. The team reviewed the summaries for themes used to describe the current situation and suggestions for addressing equity issues. At the conclusion of the visits and follow-up phone interviews, the full set of interview responses were reviewed for additional conclusions and themes that complemented, extended, or explained quantitative findings and suggested paths to consider for recommendations. Parent Survey The research team designed a parent survey to gather information on perceptions of AAP services, specifically Local Level IV and Level IV Centers, and awareness of AAP policies and procedures deemed most relevant to equity. Primarily, we were interested in parental understanding of identification policies and procedures, parental feelings about how well those procedures were working to identify students for Level IV placement, and parent feedback on how Level IV services could be diversified or improved. After initial design by the research team, the AAP office distributed a draft to several parents as well as FCPS staff for feedback. We integrated this feedback into the final survey instrument. Once a final version was established (see Appendix), we created seven copies and forwarded them to the FCPS central office for translation into the following languages: Farsi, Amharic, Spanish, Korean, Arabic, Chinese, and Urdu. On February 14th, all eight surveys were distributed by FCPS. Reminders were sent out on February 21st and 27th before the survey was closed the morning of March 2nd. As of March 2nd, each survey had the following number of responses: Amharic: 5, Arabic: 17, Chinese: 64, English: 6,071, Farsi: 2, Korean: 127, Spanish: 154, Urdu: 3. We integrated survey responses throughout the report rather than in a single section. A summary of results is included in Appendix F. Limitations Although our data collection efforts were extensive and went beyond even the ambitious plan in the accepted proposal, stakeholders should note the review team did not directly observe any classrooms – either AAP or non-AAP – nor did we speak with a large number of students. Although the plan included student focus groups, students often did not attend the scheduled meetings, limiting the amount of student input into the study. However, we believe adequate student input was received, and our triangulation of other stakeholder perspectives leaves us with a high degree of confidence in our conclusions and recommendations.

14

Literature Review on Causes of Inequity and Interventions to Achieve Equity in Advanced Academic Programs

Below we present an overview of the literature on the topic of equity and underrepresentation within gifted and talented programs. Throughout, we make connections to current FCPS policies and practices to note where they align and where there is potential for improvement. A summary of these connections is included in Table 5 on page 24. For the last 50 years, the field of gifted education has recognized and struggled to ameliorate the underrepresentation of students from non-European backgrounds in gifted education programs. Specifically, African American, Hispanic, and Native American youth have been underrepresented in gifted education services relative to their representation in the student population, while students from European American and some Asian backgrounds have been well represented or overrepresented. As most American K-12 students now identify with a demographic racial group other than European American, the discrepancy in gifted education services is a pressing matter (National Center for Education Statistics [NCES], 2017). Although perfect proportionality of all subgroups may be unrealistic given the larger state of inequality in the United States, or for that matter in FCPS, progress (or lack thereof) is important to investigate. Disparities in the identification of students who have limited English proficiency or who are served under the Individuals with Disabilities Education Act (IDEA) are also a growing issue due to these groups making up a rapidly-growing percentage of the American student population: currently 9.4% and 13% of American students, respectively (NCES, 2016a; 2016b). Sources of Disproportional Underrepresentation In a 2016 analysis, Peters and Engerrand classified the research base surrounding the causes of underrepresentation into two themes. The first theme presented the view that the assessments commonly used to make gifted identification decisions are inherently flawed or biased against certain groups, thereby resulting in disproportional underrepresentation. The response to this argument is best exemplified by the popularity of “nontraditional” assessments such as the use of non-verbal assessments of aptitude, use of specific teacher rating scales with language orientated toward the manifestation of gifted characteristics in under-represented populations; use of alternative assessment tools such as products and performances, etc. The first-grade FCPS universal screening process uses the Naglieri Nonverbal Ability Test (NNAT) – one of the most common types of such “nontraditional” assessments. From this perspective, proportional identification should be expected due to the “culture reduced” nature of the assessment, and any observed underrepresentation is due to flawed identification tools. If this is a valid claim, then the call for “less-biased” assessments for identification makes sense. In addition, if this claim is true, than identification via “culture neutral” tests should result in more-or-less perfect proportionality across student groups. Peters and Engerrand (2016) identified a second theme, suggesting the ways in which students are identified, rather than which particular assessment instruments are used, cause underrepresentation. For example, teacher recommendations as a tool may be an appropriate data source for student identification, but if such recommendations are mandatory before any other data points are considered, then their use could unintentionally exacerbate disproportionality. For example, in 2016, Grissom and Redding published an analysis of the Early Childhood Longitudinal Study – Kindergarten cohort (ECLS-K), examining the factors contributing to disproportionality in gifted education. A major finding was that African American students were far more likely to be identified as gifted in reading if they had a teacher who was also African American (6.2% vs. 2.1% probability), even after controlling for academic

15

achievement, suggesting that achievement tests themselves were not the source of the problem. Instead, a teacher-related variable was at issue. The National Association for Gifted Children (NAGC) State of the States Report (2015) noted that teacher and parent referrals are a common gate through which students must pass before being further evaluated for gifted services. The result of this two-phase identification process is that some students who would meet the identification criteria are never considered because they never receive the initial referral. FCPS does use teacher ratings, in the form of GBRS scores, in its AAP identification process. However, it does so universally, meaning all students are rated by their teachers. This is a strength and avoids one common problem with teacher ratings. However, although FCPS does conduct universal screening in first and second grade with NNAT, CogAT, and GBRS, there is a second referral pathway into the consideration pool for those who know about it. This second pathway consists of parent referrals, appeals, requests for re-testing, and procurement of outside testing. This second pathway inserts a form of assessment bias, similar to traditional uses of teacher or parent nominations, whereby those parents who know about the second pathway and take the initiative to take advantage of it, have a higher chance of their children getting identified. A growing body of research points to the use of poorly-designed two-phase identification systems as a contributor to underrepresentation (e.g., Hamilton et al., 2017; McBee, Peters, & Miller, 2016). Any time fewer than 100% of students are considered for gifted services (i.e., universal consideration), some process must be used to determine who is considered and who is not. That process is always imperfect, but certain factors contribute more seriously to bias in the identification process (McBee et al., 2016). In gifted education, the most common form of the initial screening phase is the teacher or parental referral (NAGC, 2015). If students cannot access program eligibility procedures unless first being nominated by a teacher, then simply improving the quality of the formal identification phase will never solve the problem, because many students who would (and should) be identified will never make it past the screening phase. McBee et al. (2016) gave a second example of a two-phase system wherein students must score at the 90th percentile on the phase one assessment(s) to be given the phase two assessment(s). Students must then score at the 90th percentile in phase two to be identified. Even under high reliability assumptions (.95) with a strong correlation between the phases (.90), approximately 20% of students are missed compared to universal consideration due to the existence of the conditions in this two-phase system because some students who would have done well in phase two were blocked by phase one. If the two cut scores move to the 95th percentile, just under 30% of students are missed, with most of the missed students from traditionally underrepresented backgrounds. In general, FCPS avoids many of the challenges associated with two-phase identification systems by conducting expansive universal screening of all students in grades one and two. This is a major strength over common practice across the country. However, the unequal use of referrals, appeals, re-testing, and outside testing creates a second pathway to consideration for those who know about it. We discuss this at length in the findings and recommendations. Inequality. A factor that was not addressed as a cause of underrepresentation by Peters and Engerrand (2016), which also contributes to underrepresentation, is the large inequality of educational access and opportunity in the United States. When approaching the topic of equity in gifted education, it is often implied that something close to perfect proportionality should exist, and that anything less results from flawed identification methods (e.g., biased tests or two-phase identification systems), rather than from

16

disparities in educational opportunity. Put simply, this is not a reasonable assumption given the inequality in educational opportunity in the United States. Exposure to learning opportunities influences achievement (Lohman, 2005) and measured IQ (Ritchie & Tucker-Drob, 2018). Further, in the United States, higher-income families generally have greater access to high-quality educational opportunities in and outside of schools. This is true in many countries but is especially problematic in the United States (Chimelewski & Reardon, 2016; Finn & Wright, 2015). In an analysis of nations identified as wealthy by the Organisation for Economic Co-operation and Development (OECD), Chimelewski and Reardon (2016) found that the United States had the largest levels of “poverty/inequality,” as well as some of the largest income-related achievement gaps. The United States also had the lowest parental support index, and the lowest social welfare policy index. What all of this means is that broad inequality in the United States persists and is associated with large achievement differences. Higher family income generally is associated with greater access and opportunities during the pre-school years, during the school day, and outside of formal education. Further exacerbating the effect of this inequality is that in 2015, 12% of European American children lived in poverty compared to 36% of African American children, 34% of Native American children, and 31% of Hispanic children (Kids Count Data Center, n.d.), creating an intersection of race and poverty for students who have long been under-identified for gifted education services. In a recent analysis of three states, Hamilton et al. (2017) found that even after controlling for achievement scores, individual- and school-level free or reduced-price lunch status were negatively predictive of a student’s probability of being identified as gifted. Their results make clear that individual and institutional poverty are negatively associated with the probability of a student being identified as gifted. To be clear, poverty is not the only reason for racial/ethnic disproportionality in gifted populations. However, in the United States, being African American, Hispanic, or Native American means one is far more likely to be poor and also face additional institutional barriers related to race/ethnicity. Thus, proportional representation in the absence of proactive efforts to mitigate the effects of poverty and racism on access is unlikely to occur. FCPS is often discussed in gifted education circles as a leader in talent development efforts (discussed next) to mitigate the effects of inequality on gifted education equity through its Young Scholars program. What the literature surrounding this topic makes clear is that absent efforts to mitigate the effects of poverty and unequal opportunity to learn and develop the kinds of skills taught in school, disproportionality in any advanced learning opportunity will continue. Talent Development One of the most promising frameworks with a robust intervention literature is the talent development framework (Dixson et al., 2020; Renzulli, & Reis, 1985; Subotnik et al., 2011; Worrell et al., 2019). Simply put, the talent development framework focuses on identifying and developing the academic potential of as many students as possible, with a particular emphasis on developing the academic potential of those who are highly motivated and frequently overlooked during the typical gifted and talented education identification process (e.g., a flawed two-phase identification process; Dixson et al., 2020; Olszewski-Kubilius & Steenbergen-Hu, 2017; Renzulli & Reis, 1985; Subotnik et al., 2011; Worrell et al., 2018). The goal of the talent development framework is to help students live up to their highest academic potential, both within specific domains and generally, to maximize the number of students progressing through the talent development stages—from potential to competency and from competency to expertise (Dixson et al., 2020; Worrell et al., 2019).

17

A key way that talent development frameworks deviate from other gifted education frameworks is the lack of emphasis on who qualifies for advanced academic services, typically determined by a student obtaining or surpassing an IQ and/or a standardized achievement score above the 90th percentile (McClain & Pfeiffer, 2012), and the increased emphasis on who would likely benefit from advanced academics, typically determined by a student displaying potential within an academic domain in early childhood (Dixson et al., 2020; Worrell et al., 2019). This deviation is critical with regards to equity. Students from disadvantaged backgrounds (e.g., minorities, students from low socio-economic backgrounds, English language learners, twice exceptional students) frequently encounter and must overcome a host of challenges that students from advantaged backgrounds generally do not (e.g., Ford et al., 2008; Head et al., 2019; Menken, 2010; Olszewski-Kubilius et al., 2017; Worrell & Dixson, 2018). As a result, students from disadvantaged backgrounds typically have lower academic skills at school entry and throughout schooling compared to their advantaged student counterparts (e.g., Kettler & Hurst, 2017; Reardon & Portilla, 2016). This likely leads to the typical identification process missing some academically gifted disadvantaged students since this process typically focuses on identifying students displaying high academic talent (e.g., via high IQ and/or standardized test scores), instead of high academic potential to develop (as is the primary focus of talent development programs). In sum, the talent development framework makes it more likely for students from disadvantaged backgrounds to be identified as academically talented and to have their academic potential developed. Correspondingly, talent development programs tend to display more equity in their representation of students from disadvantaged backgrounds with disadvantaged students more likely progressing through the talent development stages (Card & Giuliano, 2015; Dixson et al., 2020; Peters et al., 2019; Peters & Engerrand, 2016). Effective Talent Development Interventions Several talent development interventions have been developed and implemented to date. Overall, these interventions indicate that many disadvantaged academically gifted students are: (a) being missed by the typical gifted education identification process (e.g., a flawed two-phase identification process; Card & Giuliano, 2015), (b) would benefit from advanced services when provided the opportunity (e.g., Robinson et al., 2018), and/or (c) in many cases have similar outcomes as their advantaged peers when provided advanced services (e.g., frontloading; see Olszewski-Kubilius et al., 2017). For example, Project EXCITE is an advanced academics program designed to “support and enhance minority students’ interest and performance in math and science through extensive supplemental programing, with the ultimate goal of preparing participants for advanced-level math and science coursework in high school” (Olszewski-Kubilius et al., 2017, p. 22). Project EXCITE consists of inclusive admission criteria with the goal of growing/developing their own gifted students from disadvantaged backgrounds. They admitted African American and Hispanic students (total n = 361) who (a) scored at the 75th percentile or above on the Naglieri Nonverbal Ability Test (NNAT) or a standardized achievement test in reading or math, (b) exhibited evidence of high motivation (as measured by letters of recommendation and reported work habits), and (c) displayed evidence of academic potential (i.e., the student’s prior academic performance; Olszewski-Kubilius & Steenbergen-Hu, 2017). Project EXCITE’s programing was intensive. It front-loaded its participants starting in the third grade, providing its participants with approximately 625 hours of supplemental academic enrichment after school, on weekends, and during the summer. The results of the program are robust. After five years in the program, the African American and Hispanic Project EXCITE students outperformed their African American and Hispanic Division counterparts in both math (Hedges g = .42) and reading (g = .51), while

18

also outperforming other African American and Hispanic students that did not participate in Project EXCITE at their school in math (African American, g = 1.58; Hispanic, g = 1.21), reading (African American, g = 0.75; Hispanic, g = 0.54), science (African American, g = 1.26; Hispanic, g = 1.07), and English (African American, g = 1.38; Hispanic, g = 1.06). All effect sizes were in the medium to large range. Further, Project EXCITE students graduated and attended selective universities (de facto advanced education programs) at a rate of 50% above the national average for African American and Hispanic students and at a comparable rate as Asian American students (Ashkenas et al., 2017). Finally, Olszewski-Kubilius and colleagues reported that after five year in the project, Project EXCITE students perform similarly to European American students in math (g = -0.02), reading (g = -0.46), science (g = -0.30), and English (g = -0.31), with all differences being in the small to medium range. In addition to Project EXCITE, data from several other interventions provide evidence of effectiveness at providing increased equity within advanced education programs as well as resulting in improved outcomes for students from disadvantaged backgrounds. For example, several evidenced-based, district-specific interventions have reported increased equity and positive outcomes for students from disadvantaged backgrounds. For instance, Robinson and colleagues (2018) exposed about half their sample of 1,387 (n = 765, 55.2%) first-grade students to a challenging engineering curriculum and teachers that were trained for a week during the summer on identifying talent in students from low-income backgrounds. At the end of one year, teachers in the study referred a substantially higher percentage of African American (35.1% of total nominations) and low-income (37.5% of total nominations) students than was typical (Card & Giuliano, 2015). In addition, Robinson and colleagues reported that students participating in the intervention displayed academic gains, exhibiting higher academic performance than control group students on an out-of-level science content assessment (Cohen’s d = 0.28) and engineering knowledge test (d = 0.66). This intervention is similar to several other gifted education interventions around the country that have reported positive results (e.g., Harradine et al., 2014; Horn, 2015). Use of Local Norms Calculation and use of local norms is another intervention consistent with the talent development framework that has been found to increase equity within gifted programs as well and has been identified as a means to attaining positive outcomes for students from disadvantaged groups (Dixson et al., 2020; Peters et al., 2019; Peters & Engerrand, 2016). Standardized assessments, which include many of the tests that are used to assess and identify students for gifted programs (e.g., IQ tests, state achievement tests, the SAT), are normed on a nationally representative sample. This means that when students take the assessment, their score is compared to a national sample of students (Matthews & Peters, 2018). National norms are the default because they are useful in several situations such as when there are a limited number of college admission slots available or a limited number of merit-based scholarships to award. More succinctly, national norms are typically most useful when one is serving and/or comparing students from a national pool of candidates. National norms are significantly less useful for identification for advanced academic program services, as they are not informed by local information—the district resources, district programs offered, and the match between specific students’ needs and district resources (Matthews & Peters, 2018). If a school district has the capacity to develop the academic talent of 20% of its students, why would they limit themselves to serving only the 5% that have an IQ of 130? In addition, although an IQ at the 99th percentile nationally may indicate that a student has the ability to efficiently complete advanced cognitive tasks, it does not indicate whether that student is or has been being challenged academically

19

in his or her classes nor does it indicate whether the student is a fit for the district’s advanced programs or particular services offered. Local norms, in comparison, likely provide district decision-makers with meaningful information that is informed by the local context. Moreover, district gifted education programs do not serve the students from the entire nation, they serve the students within their school district. As a result, utilizing local norms would provide a school district information about the academic abilities of its students (relative to other students within the district), which could be subsequently matched with district services and programs. More specifically, local norms can help determine which students within a school district or school building would most likely benefit from their specific advanced services given the available resources and programs (Matthews & Peters, 2018). For example, a school district may only have the resources to develop the advanced mathematical talent of 15% of its students. If this district utilized local norms, they could easily identify the 15% of students with the highest math potential within their district. Thus, matching academic talent/potential with available district services and resources. Empirically, local norms have been found to create more racial equity within gifted education programs. For example, Peters and colleagues (2019) examined the relationship between local norms and demographic representation in advanced education programs. Their sample consisted of over 3 million students in third through eighth grade across ten states. They found that when district or school local norms were used for comparison, African American and Hispanic student representation in the top 15% of students in both mathematics (% increase ranged from 52%-111%) and reading (% increase ranged from 54%-99%) meaningfully increased while European American representation remained the same (% change ranged from -7% to -13%) and Asian American student representation meaningfully decreased (% decrease ranged from -19% to -26% in math and -20% to -27% in reading). These findings highlight the importance of implementing local norms and have been argued to relate to positive outcomes for students from disadvantaged backgrounds. Peters et al. highlighted the problematic decision to not implement local norms, as it resulted in thousands of African American and Hispanic students not being identified as academically gifted, thus decreasing their opportunity to readily access an appropriate curriculum to develop their academic potential. Had the districts utilized local norms, more students from these backgrounds likely would have had a better chance at a positive outcome. Similar arguments have been made by other scholars (e.g., Plucker et al., 2017; Worrell & Dixson, 2018). The application of local norms would identify, at the least, a slightly different group of students than would national norms. Local norms allow for identification of those who score highest in their school building compared to those who score high compared to the rest of the nation. Peters and colleagues (2019) responded to the dichotomy set between these approaches with a “compromise” plan whereby students are identified if they meet the local norm criteria OR the national norm criteria. Although this solves the problem of some high-performing students not identified simply because they attend a high-performing school when local norms alone are applied, it also increases the overall size of the population in need of services. Currently, Level IV identification in FCPS is based on a national norm. This creates an equity problem as scores at this level are often less accurate than they are when closer to average, and because students who come from less-advantaged backgrounds have had fewer opportunities to develop the skills necessary to perform at such high levels. Local norms, especially when applied to particular levels of AAP services, are an area in which FCPS could make progress toward improved equity.

20

In sum, there are several talent development interventions that have been found to increase representation, opportunity, and positive outcomes for students from disadvantaged backgrounds. Nonetheless, no matter the intervention, the elements of a successful talent development intervention are similar—increased opportunity and appropriate challenge. Strategies to Increase Equity Within Advanced Academic Programs In addition to specific interventions noted above that have been implemented to increase representation and outcomes of students from disadvantaged backgrounds, there are several other best practice strategies that have been found to increase equity within advanced academic programs. These strategies include the use of multiple criteria for identification, frontloading, and universal screening. Frontloading and student support. Gifted education is usually conceptualized as identifying academically gifted students and subsequently providing them services to develop their academic talents. A different way to view gifted education within schools, that is consistent with the talent development model, is to universally frontload students with early learning opportunities. Frontloading students means preemptively exposing them to advanced or enrichment programming before they are assessed or identified for an advanced education program. The goal of frontloading is to better prepare all students for their opportunity to be identified as academically gifted as well as to better prepare all students to persist in advanced programs should they be admitted (Briggs et al., 2008; Plucker et al., 2017a; 2017b). In FCPS, frontloading is best exemplified through the Young Scholars program. This strategy accomplishes its goals in two ways. First, it exposes all students to advanced and/or enrichment programming, which by definition limits one of the most adverse consequences of being from a disadvantaged background within the context of advanced education—lack of access to appropriate advanced programing to develop academic talents (Olszewski-Kubilius & Corwith, 2018; Peters & Engerrand, 2016; Reis & Renzulli, 1985; Worrell & Dixson, 2018). Second, through the exposure, it provides all students the opportunity to have their talent developed from an earlier age, likely leading to a higher portion of students from disadvantaged backgrounds performing better when being assessed for an advanced program as well as preparing these students for the work and environment inherent to many such programs (Plucker et al., 2017a; 2017b). The previous points are particularly important as students thrust into advanced programs without proper preparation are likely positioned to fail (Plucker et al., 2017a; 2017b; Weiler & Walker, 2009). The failure of disadvantaged students in advanced programming is particularly detrimental because: (a) many students from disadvantaged backgrounds are already hesitant to take advanced programming because they likely feel isolated from their same-group peers who are typically underrepresented in such courses (Francis & Darity, 2020; Kettler & Hurst, 2017), and (b) students from disadvantaged backgrounds seeing students from similar backgrounds failing to succeed in advanced academic programs is likely to negatively affect their own academic self-efficacy relative to being admitted to and persisting throughout such programs (Dixson, Keltner, Worrell, & Mello, 2018; Usher & Pajares, 2006). One particular academic paper bolsters the argument for frontloading creating more equity. Weiler and Walker (2009) discussed a high school that, despite being primarily made up of Hispanic students (62%), contained an AP mathematics class that was primarily made up of European American students (94.5%). In acknowledgement of the racial inequity, Weiler and Walker developed and implemented a frontloading intervention for freshman Hispanic students who were placed in remedial math to begin their high school math education. They offered them the opportunity to complete an accelerated math track that would prepare them and ultimately position them to take AP mathematics. The accelerated

21

math track consisted of a summer program intensive in math along with a special math schedule of courses that included many AP mathematics concepts. They reported that many students not only excitedly embraced the opportunity, but they excelled throughout their math career at the school, improving the racial equity within AP mathematics to 30% Hispanic and 70% European American. Put another way, when advanced math was frontloaded, students were better prepared for advanced programming later and were more likely to be identified as academically gifted later. This finding is similar to other findings reported in the literature which outline both the longitudinal and annual effects of frontloading within advanced academics and the academic context more broadly (e.g., Cuba, 2020; Olszewski-Kubilius & Steenbergen-Hu, 2017). In sum, frontloading advanced academic material allows those from disadvantaged backgrounds to have greater access to and success within gifted programs. Studies of the success of historically under-represented populations in Advanced Placement courses have also documented the importance of strong support systems for success of under-represented students in these advanced-level courses (Hanover Report, 2016; Kyburg et al., 2007). Universal Screening. One of the most common criticisms of gifted education and typical identification processes is that students from disadvantaged backgrounds are not given an equal opportunity to demonstrate their academic abilities as are those from advantaged backgrounds due to factors beyond their control (e.g., racism, economic challenges; Assouline et al., 2017; Elhoweris et al., 2005; Ford et al., 2008; Grissom & Redding, 2016; Olszewski-Kubilius & Corwith, 2018; Worrell & Dixson, 2018). For example, Grissom and Redding (2016) conducted a study examining the predictors of gifted assignment in a nationally representative sample of over 10,000 students. They found that even after controlling for standardized test scores in reading and mathematics, socioeconomic status (SES), and gender, African Americans were about half as likely as European Americans to be referred for a gifted education assessment. Similarly, McBee (2006) found in a sample of over 700,000 students that European American (5.83%) and Asian American (9.69%) students were at least 2.75 times more likely as their African American (1.96%), Hispanic (1.36%), and low SES (1.95%) counterparts to be referred by a teacher for an assessment. Many scholars have responded to this documented shortcoming by asserting that to increase equity and close this opportunity gap, advanced academic programs should universally screen all students in the early grades to both provide them the opportunity to demonstrate their academic potential as well as provide that opportunity before the effects of being from a disadvantaged background start to mount (Brown & Abernethy, 2009; Olszewski-Kubilius & Corwith, 2018; Plucker et al., 2017; Worrell & Dixson, 2018). Calls for universal screening are bolstered by studies indicating that universal screening results in more equity across demographics within advanced academic programs. Universal screening “works” by not relying on parental initiative or teacher referral. Instead, all students are tested. However, this only addresses the cause of missed students related to not being referred. It does not address any students who are missed because of larger, fundamental inequality or lack of opportunity to develop the kinds of skills the tests are measuring. That is a problem only frontloading can address. Card and Giuliano (2015) conducted a seminal universal screening study of a school district that consisted of a diverse group of roughly 40,000 third grade students across 140 different elementary schools. The district universally screened all second graders with the NNAT. All students who obtained an NNAT score that surpassed the predetermined cutoff score for the district were referred for a complete gifted education assessment (full IQ test) by a school psychologist. Card and Giuliano reported that their universal screening intervention resulted in a 67 percent increase in the total number of students identified as academically gifted throughout the district (increasing the total percentage from 3.6% to 5.3%), with the overwhelming majority of the additional students identified coming from

22

disadvantaged backgrounds. More specifically, African American student representation in the district’s gifted program increased by 145%, Hispanic student representation increased by 171%, low SES student representation increased by 75%, and English language learner representation increased by 150%. It is important to highlight that all the screening intervention did was provide students the opportunity to be subsequently assessed by a district psychologist, the criteria to be referred for advanced testing was not changed nor was the criteria for being identified as academically gifted. The screening intervention provided access, which resulted in more equity. In sum, universally screening students for academic talent allows all students the opportunity to demonstrate their gifted abilities, typically resulting in more equity across demographics within advanced academic programs. Psychosocial Interventions One of the current focal points of intervention work in education involves psychosocial interventions. Psychologists have long posited that motivation, attitude, and the social context of the learning environment are all strongly predictive of success (CPSE, 2015). Currently, interventions involving constructs such as stereotype threat, grit, mind-set, and self-affirmations receive significant attention in education. The concept of stereotype threat has been considered an especially promising area of inquiry. In essence, stereotype threat occurs when individuals within a group begin to believe stereotypes about its members’ abilities and characteristics (Steele & Aronson, 1995). Such stereotypes may limit the performance on measures of aptitude and achievement and academic success of, for instance, a poor Hispanic student if she internalizes any stereotypes about the ability of poor Hispanic females to perform at advanced levels of achievement. If this student does not see advanced achievement as something that Hispanic students “do” or “are,” then closing excellence gaps will be nearly impossible. This parallels work to help women overcome stereotype threat related to advanced achievement in science and mathematics. For over a quarter century, researchers have pointed to negative gender role stereotypes as a possible cause of female underachievement in STEM areas. Research on stereotype threat and effectiveness of interventions to counter stereotype threat is still not conclusive (Flore et al., 2018; Pennington et al., 2016; Spencer et al., 2016) particularly as it might affect particular subgroups of students. The potential role of grit, growth mind-sets, positive self-beliefs and other constructs in promoting advanced achievement and equity has been equally enticing to educators, which is probably one reason why these constructs continue to be discussed with such enthusiasm in the educational excellence research community (Clinkenbeard, 2012; VanTassel-Baska, 2013). Furthermore, although the potential overlap among all of these psycho-social constructs and their effectiveness is still subject to debate (i.e., they are currently not well-defined), non-cognitive factors clearly matter in the development of talent (Abuhassàn & Bates, 2015; Duckworth et al., 2007; Rimfeld et al., 2016). Despite the potential of psychosocial interventions, research on these constructs have not provided helpful information for equitably promoting advanced academic achievement. Part of this issue appears to be the difference between correlation and causation – highly successful students tend to have high levels of grit, for example, but it may be the case that people who are successful tend to see themselves retroactively as having been “gritty,” so to speak, when the actual grit construct had little to do with their achievements.

23

In one of the more exhaustive research programs to date, researchers studied the effects of a brief self-affirmation in several cohorts of students in a diverse school, beginning in seventh grade (Cohen et al., 2006; Cook et al., 2012). In the original study and some follow-up research, they found convincing evidence that such interventions closed racial achievement gaps (as measured by student GPA), with African American students benefiting from the intervention much more than white students. The researchers also found evidence that starting the intervention earlier in the school year yielded significant benefits. But the academic benefits were largely experienced by low-performing and moderate-performing students, not high-performing students. This suggests that such an intervention could help close minimum-proficiency gaps, but not equity gaps in advanced programs. In another follow-up study, the researchers took this work further, investigating the extent to which the intervention influenced the achievement of students who did not receive it. They hypothesized that the greater the “treatment density” within each classroom, the greater the spillover effects for all students in the class (Powers et al., 2015). And that is the limit of what they found, although treatment density for African American students appeared to be the primary driver of any positive effects. But again, low-performing students appeared to benefit significantly more than high-performing students. This research and related studies call into question whether psychosocial interventions are promising approaches to addressing advanced academics and equity in such programs, partly because stereotype threat may be less of an issue for high-potential students (Nguyen & Ryan, 2008; Yeager & Walton, 2011). Plucker and Peters (2016), in a review of the psychosocial research literature, concluded that psychosocial interventions are not suitable for routine use in schools, and that they may not work at all regarding the promotion of advanced achievement in equitable ways. Since their review, the research has turned significantly more negative, with major studies and reviews finding mixed to little evidence that psychosocial interventions provide learning benefits for any group of students (Burgoyne et al., 2018; Burgoyne et al., 2020; Burnette et al., 2020; Gandhi et al., 2020; Mehta, 2015; Moreau et al., 2019; Sisk et al., 2018).

24

Summary The interventions reviewed above are present in varying degrees in the Fairfax County Public School division. Table 5 summarizes this information as an introduction to the following sections on Findings and Recommendations. Table 5. Research-supported Interventions for Equity in Advanced Programming

Intervention Present in FCPS? Related

Recommendations

Professional development on advanced learning and identification for classroom teachers

High-quality PD available, but decentralized administrative structure makes access to this PD infrequently utilized and limited in some regions and schools

2c, 4a-4e

Universal screening All students screened at two different grade levels with multiple measures

2a, 2c, 2d

Use of local norms The division does not use local norms systematically

3b

Identification procedures that minimize advantages of socioeconomic status

The division has made a sincere effort to minimize bias in its identification procedures, but some potential sources of bias remain such as via the appeals process, requests for re-testing, and in the procurement of outside testing

2b, 2e-2h

Frontloading

The Young Scholars program is intended to provide frontloading, but its implementation is inconsistent throughout the division and limited in intensity

3a, 3b

Psychosocial interventions (e.g., mindset, grit, stereotype threat reduction interventions)

Generally NOT supported in the research literature, but some FCPS leaders talked about the usefulness of such interventions for advanced learning

N/A

25

Major Findings and Conclusions This section highlights our major conclusions from the study. Readers should note that this section focuses on the contracted task – how to improve equity in FCPS advanced academic programming – and not a broader set of issues on which we received input. For example, the complications caused by Level IV Centers within the FCPS pyramid system were often shared with the review team, as were principal concerns about how state achievement test results are attributed to schools for Level IV Center students. These and many other issues are important and worthy of being addressed, but they do not directly relate to the assigned project. 1. AAP programs in FCPS meet the relevant statutory requirements of the Commonwealth of Virginia

and are considered a net positive by the community.

FCPS programming appears to meet the requirements for advanced programs described in Virginia Code (The Appendix contains relevant sections of the Commonwealth of Virginia Code regarding gifted education).

AAP is considered a net positive by the community and a key defining feature of FCPS by many parents and outside stakeholders. Stakeholders described AAP programming as key to the “identity” of FCPS, sharing stories of how local businesses use the presence of AAP programming in their effort to recruit employees.

The vast majority of respondents to our parent survey believe that FCPS should continue offering Level IV Centers as one of its AAP service options. Only 18% of parents believe they should be discontinued.

2. FCPS has taken a number of steps to increase equity of participation in its Advanced Academic

Programs:

Over the past 10-12 years, FCPS has grown the number of students receiving AAP services significantly. For example, Table 3 shows a 186% and 322% increase in African American and Hispanic identification for Level IV services over the last 10 years. This growth is not trivial and is a considerable strength, as most strategies for improving equity in advanced programs involve at least some degree of expanded programming and significant expansion of resources.

Several aspects of the FCPS talent identification procedures are sound and reflect best practices, such as universal screening, use of multiple criteria, referencing school contexts in certain stages of the screening process, having at least one Advanced Academics Resource Teacher (AART) in each Title I school, and frontloading via the Young Scholars program in many schools across the county.

Indeed, many of these features and services were noted and widely praised by stakeholders during the focus groups and interviews, especially the assignment of AARTs to every Title I school, the quality and usefulness of AAP professional development offerings, universal screening, and the philosophy behind the Young Scholars program.

The staff of the AAP Program have a demonstrated commitment to identification of and provision of appropriate services to twice-exceptional learners as evidenced by their contributions to the Fairfax County Public Schools (FCPS) Twice-Exceptional (2e) Handbook and ongoing professional development opportunities.

There is considerable face validity for many of the current efforts and activities. However, face validity (i.e., when an activity looks to be reasonable, justified, and productive), does not necessarily serve as evidence that the programs are working as desired. Indeed, the review team found

26

evidence that the division’s strategies and programs are not working as well as they could be. In the remainder of this section, we share our conclusions about the identification of advanced students within FCPS, the AAP programming delivered to advanced students, and FCPS’ staff expertise and professional development. Findings Related to Identification 3. All three of the major data points collected via universal screening (NNAT, CogAT, GBRS) show lower

scores for African American and Hispanic students than Asian American and European American students. Table 6 provides descriptive statistics for the GBRS, NNAT, CogAT-NV, and CogAT Composite. These mean-score gaps are similar to those seen nationally and on other tests, such as the National Assessment of Educational Progress (NAEP). These data suggest no single test is the barrier to greater equity and that larger societal inequality is likely manifesting in lower mean scores for certain subgroups.

Table 6. Mean Scores for Universally Administered Assessments by Student Race/Ethnicity

Scale White Black Hispanic Native

American Asian

Hawaiian/Pacific Islander

Two or more

Total GBRS 10.5 9.7 9.4 9.9 11.0 9.9 10.8

NNAT 107.5 97.6 98.3 102.0 113.6 102.4 108.9

CogAT non-verbal

110.8 101.2 100.8 103.0 114.8 106.9 112.1

CogAT composite

112.1 101.2 99.6 104.1 115.7 105.6 113.4

Note. Full statistics included in Appendix D.

4. One way to evaluate bias in an identification system is to compare one group's odds of being identified, compared to another group's, while controlling for test scores. In doing so we effectively compare apples to apples - students who received the same test scores but only differ from each other in terms of their race or ethnicity. This is an inherently imperfect process as there are a host of ways students differ that aren't captured by this analytical method, but it is still useful to see if two students who scored the same were otherwise treated differently when it came to the Level IV identification process. For example, were African American students who scored 130+ on NNAT still considered for Level IV services at the same rate as their European American peers? Were they identified as eligible for Level IV services at the same rate? These are the questions we tried to answer. In the end, we found no evidence of racial / ethnic bias against African American or Hispanic students in the Level IV selection process (either at the screening or eligibility phases). In fact, Tables 7 and 8 shows that African American and Hispanic students had far greater odds of being screened (Table 7) or found eligible (Table 8) for Level IV services than their similar-scoring (on GBRS, NNAT, and CogAT) European American peers. Asian American students also had greater odds of being screened for or found eligible for Level IV services than their similar-scoring, European American peers. We also found no evidence of gender bias at the screening or Level IV eligibility phases.

27

This means that although African American and Hispanic students are still disproportionally underrepresented in Level IV services, they are actually being placed in Level IV services at higher rates than would be expected given their NNAT, CogAT, and GBRS scores. African American students with similar scores as their European American peers were identified for Level IV services at a rate of 5.7 to 1. Similarly, while Asian American students are disproportionally overrepresented in Level IV services compared to their enrollment in the overall FCPS population, this rate is close to what should be expected given their test scores. Asian American students with similar scores as their European American peers were identified for Level IV services at a rate of 1.1 to 1 - almost identical. When we look at the NNAT alone, we still see no evidence of a bias against African American or Hispanic students. The NNAT is described by the publisher as a "culture neutral" test of general ability. When we control for NNAT scores, African American students were still found eligible for Level IV services at a rate of 1.5 to 1 - less than when we control for all available data, but still no evidence of any negative bias against African American students. In short, this suggests that much of what FCPS has been doing to increase the diversity of Level IV services has been working. If Level IV services were based on CogAT, NNAT, and GBRS scores alone, far fewer African American and Hispanic students would be identified as Level IV eligible than they are today. Why is there such a disparity? It is likely due to larger inequality and the achievement gaps shown in Table 6. Nationally, students enter school with different levels of content mastery and readiness. These achievement gaps across groups exist on nearly every test and when any of those tests are used for program admission, these disproportional enrollment figures should be expected. Currently, it appears that FCPS is putting forth significant effort to increase the Level IV eligibility rate of African American and Hispanic student, but the achievement gaps between groups is so large that this effort is still not enough to make the Level IV population reflective of the overall student population.

Table 7. Logistic regression for race / ethnicity

Wald Test 95% Confidence

interval

Parameter Estimate Standard

Error Odds Ratio

z Wald

Statistic df p

Lower bound

Upper bound

(Intercept) -20.132 0.434 1.806e -9 -

46.423 2155.135 1 < .001 -20.982 -19.282

CogAT composite

0.107 0.004 1.113 28.502 812.340 1 < .001 0.100 0.115

Total GBRS 0.405 0.012 1.499 33.560 1126.258 1 < .001 0.381 0.428

Race (Black) 0.627 0.118 1.871 5.305 28.140 1 < .001 0.395 0.858

Race (Hispanic)

-0.035 0.090 0.966 -0.385 0.148 1 0.700 -0.212 0.142

Race (Asian) 0.487 0.079 1.628 6.159 37.934 1 < .001 0.332 0.642

Race (Two+) 0.058 0.119 1.059 0.485 0.235 1 0.628 -0.175 0.290

NNAT 0.022 0.003 1.022 6.890 47.466 1 < .001 0.015 0.028

Note. Screened for Level IV? Y/N level 'Y' coded as class 1.

28

Table 8. The impact of race / ethnicity on Level IV eligibility Wald Test

Parameter Estimate Standard Error Odds Ratio z Wald Statistic df p

(Intercept) -33.703 0.867 2.306e -15 -38.870 1510.867 1 < .001

Total GBRS 0.638 0.019 1.893 33.614 1129.933 1 < .001

CogAT Q 0.090 0.005 1.094 18.662 348.267 1 < .001

CogAT NV 0.045 0.005 1.046 8.403 70.607 1 < .001

Race (Black) 1.740 0.166 5.696 10.480 109.821 1 < .001

Race (Hispanic) 1.335 0.129 3.800 10.375 107.646 1 < .001

Race (Asian) 0.106 0.104 1.111 1.011 1.022 1 0.312

Race (Multi) 0.122 0.149 1.130 0.818 0.669 1 0.414

Cog AT V 0.046 0.005 1.047 9.644 93.003 1 < .001

NNAT 0.025 0.004 1.025 6.268 39.284 1 < .001

Note. Level IV Eligible level 'Y' coded as class 1.

5. Any measures that insert parental advocacy into the identification process will bias that process in favor of parents from certain demographic groups. For example, as noted above, students are placed into the consideration pool based on their CogAT / NNAT test scores or based on a referral. This means that if a child did not receive high enough test scores to be considered for Level IV, his or her parent can refer him or her directly. Table 9 includes the rates at which students were considered for Level IV services, by race and ethnicity, based on which pathway they took – testing (pool) or referral. As Table 9 makes clear, for last year’s grade 2 cohort, more students were considered for Level IV services based on parental referral than on all forms of testing (CogAT and NNAT), and by far, European American and Asian American families are those who utilize the referral pathway most often.

Table 9. Pathways to Level IV Consideration: 2018 – 2019 Grade 2 Cohort

Ethnic Pool

Candidate Parent

Referral Teacher Referral

Self-Referral

Total

White 596 1044 114 0 1754

Black 26 179 61 0 266

Hispanic 72 199 139 0 410

American Indian 4 5 0 0 9

Asian 592 555 79 1 1227

Two or more 118 181 23 0 322

Native Hawaiian 1 2 0 0 3

Total 1409 2165 416 1 3991

The referral option is not the only place where parents can insert initiative or cultural capital. Two other examples are the option to request a retake (of CogAT or NNAT) and the ability to appeal any negative decision about Level IV eligibility. Re-takes of NNAT or CogAT are far less common than appeals. However, they still show significant racial / ethnic gaps. From 2014 – 2019, only 2.2% of

29

students took the CogAT more than once. However, 72% of re-takes were by students from Asian American or European American families.

6. An appeals process for placement for gifted services is mandated by Virginia law. In 2018 – 2019, 46% of students screened for Level IV were found ineligible (n=3118). Of those, 19% decided to appeal the decision (n=587) – 87% of whom were from Asian American or European American families. But any appeals process serves as a pathway to identification solely for those who know about it. For example, only 46% of parents who completed the Spanish version of the survey reported they were aware of the option to appeal a negative Level IV decisions, compared to 71% completing the Korean version of the survey, and 80% completing the English version2 It is important to note that all eligibility decision letters sent home to families include information about how to appeal a negative decision. The appeals process also biases the AAP identification process in favor of families that have the time and means to pursue it. Given that the process mandates that the appeal includes additional information not considered previously, those that pursue it must have the time, resources, and motivation to pursue and complete the collection and inclusion of the necessary additional information. The process also seems to encourage additional test data, obtained at parents’ expense, although AAP staff communicate that additional test data are not necessary for a successful appeal. This appeals option likely exacerbates disproportional representation in Level IV services

7. Other examples of policies that allow for parental influence in the identification process include allowing parents to include test data from a private psychologist hired at the parents’ expense and allowing for a range of optional items to be included in the student’s portfolio, many of which are influenced by socioeconomic status. Some of these are discussed at greater length below.

8. During parent focus groups, many participants indicated that many parents are unaware of the purpose and goal of AAP and thus do not know the importance of attending information sessions or helping to complete portfolios. Although some recommended including information about gifted programming options and processes in the folders students take home from school, others noted that the advantage inherent in parent input suggests that the option be eliminated entirely. This all points to the importance of parent and family engagement (and potential for bias when it is cannot be achieved), especially when there are referral pathways to AAP services.

In summary, although some of these identification policies could help catch students who would benefit from Level IV services, almost without question, having them in place will also exacerbate the inequity of the Level IV service population. 9. The Level IV application requires two student work samples from school work. This would be an

acceptable practice if all students had an equal opportunity to be assigned projects or tasks that would result in high-quality artifacts. If a student attends a school where there are lower

2 Of parents who completed the Spanish version of the survey, 74% indicated their child received lunch

assistance, 24% that their child was twice exceptional, 11% reported speaking English at home; for the Korean version of the survey, 14% lunch assistance, 16% twice exceptional, 9% English at home; for the English version, 85% White, 5% lunch assistance, 12% twice exceptional, 90% English at home.

30

expectations or where teachers are not providing assignments that prompt students to demonstrate critical thinking skills, then a lack of artifacts that show those skills may not be an indicator of lack of ability, but rather lack of opportunity.

10. “Certificates, honors, or awards” are allowed on the application / Level IV portfolio. Although students may have universal access to receive “awards” for school-related activities, there are countless activities that students can only access if parents can pay. Examples include Talent Search certificates, awards for participating in extracurricular activities, or even sports trophies. Parents who can pay for their child’s participation in those activities and can also afford to pay for talent development in those activities (e.g., private coaching in sports or instruction in music lessons) have an advantage on this component.

11. Although outside awards and materials are allowed as part of the application (see above), teachers are prohibited from including materials aside from their ratings of the student on the GBRS (see AAP handbook section on screening file preparation guidelines). This prohibition again disadvantages students for whom the classroom teacher is a primary advocate. The teacher of a student whose primary or sole opportunity to demonstrate potential is the K-12 classroom is prohibited from including additional materials (e.g., a letter of recommendation) even though that professional is in the best position to observe the student’s potential. Note that teachers are allowed to refer students for all levels of AAP service (in 2018 – 2019 teachers referred 387 second-grade students into the Level IV consideration pool – the plurality of whom were Hispanic).

12. The review team heard multiple accounts of teachers being asked by parents to “re-do” a GBRS to arrive at a higher score. Although we have no quantitative data on the prevalence of this phenomenon, principals, teachers, and parents all recounted particular instances where requests had been made and met with positive response. This action biases the process in favor of students from some demographics whose parents feel more comfortable advocating for a redo over others who generally accept the results of the first GBRS. The review team was also told by several stakeholders that parents receive a copy of the GBRS, which would appear to open the door for assertive parents to contact teachers about individual scores. Although the identification procedures would prohibit sharing of GBRS scores or portfolio information, there appear to be some individuals who disregard the guidelines. Sharing of scores in any way will ultimately affect teachers’ willingness to provide unbiased student evaluations.

13. GBRS scores are more-strongly associated with a student being found eligible for Level IV services

than NNAT or CogAT scores. For example, if two students have identical CogAT and NNAT scores, but one student had a higher GBRS score, that student has twice the odds of being found Level IV eligible. Compare this to two students who have identical NNAT and GBRS scores, but one student has a higher CogAT-Q subscale score. Despite that higher CogAT-Q score, the two students still have the same odds of being found Level IV eligible. Based on the provided data, GBRS scores are more-influential of Level IV eligibility decisions than are scores on the CogAT or NNAT. If GBRS scores are not a valid measure of student potential and/or achievement or they are inflated because of parental influence or fear of parental reprisal, then their influence on Level IV eligibility is negative. However, if GBRS scores capture some information that is essential in locating which students will benefit most from Level IV services, then their inclusion and influence are positive. Based on existing information, there is no way for the review team to know with any degree of certainty that the scores on the GBRS is a valid indicator of eligibility for gifted services.

31

14. Other forms of cultural and socioeconomic capital are indirectly related to the screening process. For example, one parent shared a story about teachers allocating bonus points to students’ grades when parents responded to teacher emails, which potentially affects students’ report card grades. Furthermore, some parents noted that transportation to AAP information sessions (e.g., centered around the identification process and placement options) is an issue for some parents. The AAP department noted that in 2019 – 2020 expanded information on the screening and placement process was added to the FCPS website for parents who could not attend information sessions. The challenge remains in ensuring that parents would know the importance of seeking such information and would know where and how to access it. Parents who do not have the opportunity to participate in these sessions lack the knowledge of their role in helping prepare the students’ portfolios. Even those who can attend may not have equal time or resources to provide the same level of assistance. Another emergent theme around transportation issues was the finding that some parents choose not to send students to centers because they do not have transportation to the school events that their children participate in and/or to meetings such as PTO and back-to-school events. Transportation issues were also identified in the parent survey results as well when parents commented on the challenge of having multiple children attending multiple school sites. Although the review team could not confirm the accuracy of these anecdotes, these issues came up frequently enough to warrant their mention here as they could inject considerable self-selection bias into the system.

15. As described above, if a student is not placed in the consideration pool for Level IV services, based on the first and second grade screening tests, then the student may be referred into the consideration pool. This privileges students with highly-engaged parents or other advocates who are aware of this pathway and are motivated to pursue it. In addition to the equity problems this creates, it also considerably increases the administrative burden for the AAP staff as they administer the Central Selection Committee process.

Tables 10 and 11 provide the number of students screened for Level IV services (i.e., the number of portfolios reviewed by the Central Selection Committee) and the number of student referrals made by year. Correlating the two sets of numbers shows that the correlation between number of files screened and number of referrals is .86. This suggests the largest factor in work load for the Central Selection Committee is the growth in students being referred for consideration despite not scoring high enough on NNAT or CogAT to warrant such consideration. Put differently, the identification system has a universal screening component, but there are numerous aspects of the system that make the universal screening less important than intended, at least for certain populations of students.

32

Table 10. Central Selection Cycles - AAP Screening 2008-2018

Screening Cycle Eligible Ineligible Total Files

Screened

2007-08 2,604 1,568 4,172

2008-09 2,654 1,754 4,408

2009-10 2,615 1,602 4,217

2010-11 2,751 1,606 4,357

2011-12 3,409 1,775 5,184

2012-13 4,222 1,869 6,091

2013-14 3,618 2,022 5,640

2014-15 3,879 2,713 6,592

2015-16 3,807 2,139 5,946

2016-17 4,249 2,638 6,887

2017-18 4,228 2,579 6,807

2018-19* 3,572 2,679 6,251

Source file: 2008-2018 comparisons v2 Table 11. Students Referred for Level IV Consideration: 2010 - 2019

REFERRAL White Black Hispanic American

Indian Asian Two or more

Native Hawaiian Undesignated Total

Spring 2010 1051 77 80 3 726 101 1 5 2044

Spring 2011 1229 146 197 3 833 139 12 2559

Spring 2012 1512 200 257 11 1016 165 2 21 3184

Spring 2013 1657 262 241 2 1080 201 1 12 3456

Spring 2014 1822 320 365 7 1197 274 4 3 3992

Spring 2015 1630 312 386 12 1101 254 2 3697

Spring 2016 1958 445 738 16 1429 288 3 4877

Spring 2017 2049 405 628 13 1392 308 7 4802

Spring 2018 2054 447 580 14 1358 340 5 4798

Spring 2019 1935 400 637 9 1294 336 2 4613

16. A further source of potential bias against students who are still learning English (ELL) comes from the characteristics of the assessments used including written or spoken directions – most often in English. Although the NNAT is comprised of non-verbal items and no language is included in the items themselves, directions are still given to students in English before they start. These directions are supposed to be conveyed in whatever language is appropriate for the students, but we imagine the instructions are most often given in English. Similarly, although most items on the CogAT administered at the 2nd grade level are nonverbal and are based on pictures, figures, or graphics, the directions are still administered in English in FCPS. CogAT and NNAT are two of the most bi-lingual / ELL friendly assessments on the market. However, they do still involve and measure some level of English language proficiency. Given the diversity of languages spoken at home for FCPS students, we

33

concur with recommendations that assessments be administered in a student’s native language whenever possible and appropriate.

17. The Central Selection Committee makes the final decision about a Level IV placement. The committee makes its decisions based on a range of criteria included in the screening file. This is an inherently subjective process, which has strengths and weaknesses. A strength lies in the inclusion of multiple data points for consideration. However, there are several weaknesses. As a case in point, the review team heard from individuals who have served on the Central Selection Committee that a student’s eligibility often depends on who reads and rates the file. In addition, some principals believe that including the school name on student folders during the selection process influences decision-making (a double-edged sword in that this may be beneficial when the reader understands issues of cultural capital and detrimental when the reader has a negative cultural bias). The identification process is seen by some stakeholders as trying to “fit kids into a program” rather than designing educational interventions that meet the needs of students. If special education services, including those for the gifted, are designed in response to differing learner characteristics which are not adequately served in the traditional classroom, then all students whose needs are not met in the classroom should have the opportunity for services, with consideration being given to both their cognitive and peer group needs.

18. The review team saw little opportunity for “late bloomers” to enter into AAP Level IV programming,

which can be a particular impediment for low-income students, ELL students and twice-exceptional

students whose giftedness may have masked identification for special education services and whose

disability may have masked eligibility for gifted services. Specifically, the review team heard from

several school-based stakeholders (e.g., teachers, AARTs) that the emphasis on finding advanced

learners was a much higher priority in elementary schools – particularly at the primary level – than

middle schools.

Many middle school stakeholders (e.g., teachers, AARTs, and principals) recommended there be a

trial period for students in AAP, as many of the students in middle school AAP are perceived to

struggle. Stakeholders felt there should be a way to exit a student from the program that is fair and

equitable (and still provides the student with appropriate challenge when exited).

19. Inconsistency regarding how students were identified for Level III services was frequently

highlighted across many different stakeholder groups (e.g., parents, teachers, principals). Multiple

stakeholder groups also identified inconsistencies in how decisions are made by school personnel

for placement of students identified for Level III services into the school’s local Level IV classroom.

These stakeholders noted inconsistencies from year to year and from school to school, with some

stakeholders noting that they used a formal process and meeting to make such decisions including

input from multiple sources (teachers, principals, parents, etc.) while others indicated a single

identified decision-maker was responsible for such decisions.

20. Identification as eligible for Level IV services and enrollment in one of the Level IV Centers is

perceived by many parents as the pathway to admission to Thomas Jefferson High School for

Science and Technology (and to subsequent admission to elite colleges, universities, and careers).

Many stakeholders noted the early acceleration in mathematics at Centers leading to enrollment in

34

advanced mathematics in middle school is a major factor in admission to the selective high school.

Over the past years, nearly all of the FCPS students admitted to Thomas Jefferson High School for

Science and Technology were Level IV Center eligible students, suggesting the issue of parental push

identification for Level IV services is very complex and not just a matter of seeking “status” in the

community as some stakeholders suggested.

21. Many middle school parents reported that they do not push for their children being in AAP in middle

school, because honors classes are open enrollment and they feel there is no point in pushing for

AAP when their children are already taking three honors classes. Nonetheless, even if their child is

talking four honors classes (the middle school equivalent of local Level IV), other stakeholders

believe AAP in middle school is a huge step up in rigor and challenge from honors courses. We did

not examine data to see if there were equity issues in these middle school course-taking patterns,

and this issue deserves further examination.

Findings Related to Programming 22. Across all stakeholder groups, the value of Level I and Level II services was questioned. Nearly all

regarded these services as, in the words of one principal, “what a good classroom or good classroom instruction should look like.” Few stakeholders at the school-level were able to articulate exactly how AAP Level I or Level II services should be different from regular classrooms except to indicate that lessons from the AAP curriculum framework should be taught. Many principals questioned the value of offering Level II services at all and were unable to articulate the goal of those services and whether they were associated with any evidence of success in their schools. In many cases, implementation of Level II services were described as highly variable and inconsistent.

23. Some stakeholders noted that providing greater opportunities for high-end learning through critical and creative thinking by requiring the integration of lessons from the AAP curriculum in general education classes was an “admirable attempt to bring gifted curriculum to the general education classroom,” but many teachers and principals noted highly inconsistent implementation both in the way lessons are selected and how they are delivered across schools and classrooms. Several AARTs noted that general education teachers generally lack the time to use AAP curriculum in their classrooms as they have too many other competing requirements and priorities from the administration. Another stakeholder noted, “AAP curriculum cannot be successfully implemented at all levels of [the] student population. If it could be, it would not be appropriate for gifted students.” Teachers noted a degree of randomness in AAP curriculum selected for use in their classrooms.

24. The review team noted similar inconsistency in the delivery of Level III services. These inconsistencies included (but were not limited to) students not receiving Level III services every week because the AART had competing priorities, curriculum differing across students and schools (even within the same content area), and the amount of time students were given with the AART during the pullout sessions (e.g., some regularly received a full hour, others regularly participated in only 25-35 minutes). Nonetheless, across groups, there was more buy-in to the value of Level III services (compared to Levels I and II), with acknowledgement that AARTs are responsible for delivery of Level III services, and that services are provided by content area to students who excel in a particular content area.

35

25. Several stakeholders shared a sense of dissatisfaction with Center Level IV programming – due in part to racial isolation or disappointment with support for twice-exceptional students. They contended the lack of satisfaction with Centers was severe enough that many Level IV students attending Centers returned to their home schools at the end of each year. The implication was that Hispanic, Black, ELL, recent immigrant, and twice-exceptional students were more likely to leave centers than other students. However, from the 2015–2016 school year to the 2016–17 school year, out of 550 African American students served, only one deferred service. Fourteen more chose to move to honors classes at the middle school level rather than stay enrolled in a Level IV Center. Out of 785 Hispanic students, only three deferred Level IV services the next year and 20 moved to honors classes. Similar trends can be seen for students served by Level IV Centers and under IDEA as well as students who are still learning English. There is little evidence to support the claim that significant numbers of students from traditionally disadvantaged groups are leaving Level IV Centers once identified (Table 12).

36

Table 12. Level IV Continuing Students: 2015 – 2016 to 2016 - 2017

Student attending Level IV in

Centers grades 3-7

Continuing in Center ES

MS

Moved to Local

Level IV

1-4 Honors classes (MS)

Levels II-III

Deferred services

Inactive / out to FCPS

Returning Students

Remaining at a Center

Total 9921 9215 28 250 16 21 391 97.7%

Ethnic Group

White 4118 3745 13 172 10 16 162 94.7%

Black or African American 550 508 1 14 2 1 24 96.0%

Hispanic 785 718 1 20 0 3 43 96.8%

American Indian/Alaska Native 18 17 0 1 0 0 0 94.4%*

Asian 3712 3551 8 21 1 0 131 99.2%

Two or More 726 668 5 21 3 1 28 95.7%

Native Hawaiian/ Pacific Islander 12 8 0 1 0 0 3 88.9%*

ELL

ELL Level 1-10 4333 4110 10 50 2 3 158 98.4%

Non-ELL 5588 5105 18 200 14 18 233 95.3%

IEP

IEP 244 226 1 14 0 1 2 93.4%

Non-IEP 9677 8989 27 236 16 20 389 96.8%

* In both cases, attrition was one student.

37

26. Stakeholders across regions and roles shared a widespread belief that the Level IV Center model should be modified. However, there was considerable diversity of opinion on what those changes should include. In the end, the review team does not believe that most of these center-related programming issues are the cause of the AAP equity issues, nor does the team believe addressing center-related programming issues will significantly impact equity. Although some aspects of the center programming appear to need further scrutiny, they are not discussed in this equity-focused report. However, the following issues related to Level IV Centers could impact equity:

Center teachers questioned the relevance of the curriculum for students from diverse

backgrounds. They noted that the William and Mary curriculum and Caesar’s English, for

example, do not appear to be relevant to these students, and that the students don’t have the

experiences to equip them to be successful with this curriculum.

Center teachers claimed some identified students who attend the Centers are not reading even

on grade level, making it “a waste of time to assign the William and Mary readings.” Although

the review team does not have data on reading performance levels for Level IV Center students,

we were able to investigate the range of scores in the universal screening. The following tables

illustrate the variability across students who are found to be Level IV eligible (see Tables 13 and

14). For example, the tables show that some Level IV students score as low as 89 or 90 on the

CogAT-Verbal (mean score = 100). For context, a CogAT score of 90 is 2/3 of a standard deviation

below the national average score of 100. A student with a score of 90 has scored below 75% of

national second-graders (in the 25th percentile). Similarly, one FCPS student who was identified

for Level IV services with a CogAT-Verbal score of 154 has scored higher than 99.98% of national

second graders (the maximum score is 160). This shows just how wide a range of learning needs

there are within the Level IV service population – and this is just in one content area. Nearly

identical diversity of readiness was present in CogAT – Quantitative scores. For all intents and

purposes, 75% of the entire range of verbal and quantitative readiness and content mastery is

present in students who were identified for Level IV services. This is only a slightly narrower

range than the diversity of learning readiness present in the entire FCPS Grade 2 population.

Table 13. Descriptive NNAT Statistics for Level IV-Eligible Students

White Black Hispanic Native Asian Multi Haw

Valid 827 144 257 5 638 199 1

Missing 53 10 14 0 39 11 0

Mean 118.266 111.056 112.708 133.800 129.019 120.789 102.000

Std. Deviation 11.932 11.169 10.301 25.004 15.047 11.056 N/A

Minimum 70.000 85.000 85.000 96.000 93.000 90.000 102.000

Maximum 160.000 142.000 143.000 158.000 160.000 160.000 102.000

38

Table 14. Descriptive CogAT Verbal Statistics for Level IV-Eligible Students

White Black Hispanic Native Asian Multi Haw

Valid 876 154 270 5 676 210 1

Missing 4 0 1 0 1 0 0

Mean 122.120 114.266 114.085 115.400 120.587 122.714 117.000

Std. Deviation 9.722 10.130 10.552 3.912 10.192 9.291 N/A

Minimum 93.000 90.000 89.000 110.000 92.000 99.000 117.000

Maximum 151.000 143.000 139.000 121.000 154.000 150.000 117.000

Based on these and similar data, the teacher belief that some Center students are reading below

grade-level is plausible. If true, this could be due to efforts by AAP and Central Selection

Committee members to identify students who are still learning English for Level IV services.

Proactive efforts to search out additional ELL students who would benefit from Level IV services

are positive strategies, but these efforts come with the added challenge that those students will

then need to be properly supported in Level IV services. What we can say for sure is that the

students identified as Level IV eligible do not constitute a homogeneous group of learners and

many require scaffolding for success in the program—a programmatic feature that was not

readily discernable from the interview data.

27. Stakeholders spoke both favorably and unfavorably of Local Level IV services, with principals and

teachers especially believing that delivering Level IV services in local schools (i.e., outside of Centers)

was conducive to equity in participation of advanced academics. However, concerns were also

noted by several groups of stakeholders. These concerns included inconsistency across schools in

implementation of Level IV curriculum, concerns about teacher training, and the possible “watering

down” of the Level IV curriculum because of the inability of some Level III students to work at the

same pace as identified Level IV students (as mentioned above, Level II and Level III students are

commonly placed in Level IV classrooms to meet class size requirements). The review team sees this

last point as evidence of the lack of teacher ability to scaffold and differentiate for a range of

advanced performance levels. Additionally, parents of students receiving Level IV Center services

often believed that Local Level IV services were of lower quality and less rigorous.

28. The review team heard similar concerns about Honors Algebra I. As one stakeholder shared, “there

is a clear tension between the County’s desire to increase enrollment among under-represented

groups and the readiness of students to take Algebra I and their ability to learn the material well

when in the class.” This is not a concern unique to FCPS – across the state and nation school districts

are striving to challenge advanced learners while also better enfranchising students from all

backgrounds in their advanced learning opportunities.

29. The Young Scholars Program is regarded across stakeholder groups as a program based on a positive

philosophical framework for the development of talent among historically under-represented populations. However, the implementation of the program received varied reviews. It became apparent to the review team that in some schools there is a fully developed commitment to this program, with the school allocating significant resources in its implementation, making it consistent and comprehensive. In other schools, the program appears to exist in name only with the only

39

vestige of the program being assignment of a label to some students. Other, specific limitations and concerns include:

Wide variation in quality of implementation of Young Scholars (and Level I and II services)

The relatively “low dose” of intervention

Inadequate funding to cover the attendance of all Young Scholars in the summer program activities

Inadequate funding to increase the consistency of programing throughout the county

Inadequate funding/resources to make the program more robust at the middle school level

The inconsistent role of the AART within each school regarding implementation of Young Scholars. School level implementation varies widely; in some schools, the full-time AART takes care to implement YS and to create strong portfolios for historically under-represented students; in other schools, the AART is perceived to lack commitment or be so busy as to not have the time and opportunity to help Young Scholars. The assignment of AART teachers to “the master schedule” in some schools inhibits their potential to attend fully to duties related to the Young Scholars program (and Level II support).

30. A major conclusion across many stakeholder groups is that Young Scholars, Level I, and Level II services are not systemic enough in providing rigorous, challenging experiences. All are too episodic and weak doses of advanced thinking and learning and, therefore, do not have the desired impact even in those schools with great commitment and a full-time AART. Classroom teachers are not viewed as being skilled enough in differentiation to create such an environment for those with potential talent. This represents a breakdown in talent development opportunities. Young Scholars, AAP Level I and AAP Level II should be developing potential in students, but because of variable availability and implementation, they are not working as well as they could and should in this regard.

Findings Related to Expertise and Professional Development 31. Advanced Academic Resource Teachers (AARTs) are highly valued by many principals, AAP teachers,

and non-AAP teachers. This is especially true for full-time AARTs in Title I schools. Principals were especially complimentary of the teacher training provided by AARTs on using the GBRS, on implementation of Level III curricula, on using multiple instructional strategies in general education classrooms, and in providing support for Young Scholars – both in general and during the preparation of student selection packets for Level IV consideration. In particular, several principals noted that the AART at their schools had established strong partnerships with the school’s parent liaison and/or equity lead, which also was perceived to be very helpful in addressing AAP equity issues. However, stakeholders also shared several concerns about the role of AARTs:

Quality of local services – and, therefore, support for equity – appears to depend heavily on each school’s AART. Having a stronger AART was often described as a major asset for a school; having a weaker AART was considered a deficit.

Half-time AARTs were frequently described as “a waste of time and money.” AARTs work with teachers in some schools to offer the only support for Level II services, but the AARTs must also provide Level III services, are expected to support Young Scholars, and work as an instructional coach. In schools with one full-time AART, this can be difficult; in schools with a half-time AART, it becomes almost impossible (hence the widely reported inconsistency of services among schools). In one case shared with the review team, a half-time AART was expected to support 36 classroom teachers plus execute the other responsibilities mentioned above. However, for many

40

FCPS schools, picking up another .5 FTE to create a fulltime position is considered too great a financial burden given limited budgets with competing priorities.

In addition, in very large, Title I schools, one full-time AART is perceived to be inadequate for the work that needs to be done. As a result, they give up one or more functions. The review team was told this usually means the AART gives up working with classroom teachers because of scheduling and time issues.

Although AAP staff noted that professional development regarding coaching is included in AART meetings, several stakeholders expressed a desire for AARTs to have more training on how to be good coaches.

32. As noted above, principals and teachers had generally positive comments about the quality of AAP

professional development offerings. For example, principals across the regions shared that the professional development on Young Scholars helped them get a better understanding of the purpose and basic structure of that program. Both AAP and non-AAP teachers mentioned their appreciation for the PD offerings, with several recommending that all teachers should have at least some exposure to this training in order to meet the needs of students receiving services at levels I, II, and III. However, in spite of many offerings, the professional development still only reaches a small portion of teachers, particularly non-AAP teachers. In addition, even teachers who widely praised the AAP PD offerings noted that the lack of reliability of substitute teachers made even the best PD plans rather tenuous.

33. The lack of African American and Hispanic teachers and principals was often noted as a weakness for the entire school division. Recent research provides evidence that Black and Hispanic teachers are more effective at identifying advanced academic talent in Black and Hispanic students, making this an area of significant concern. Indeed, teachers were identified by parents in several focus groups as gatekeepers who don’t understand how advanced academic talent manifests in a diversity of cultures, and they indicated that teachers reportedly urge students “not to go where everyone thinks they are better than everyone else.” Teachers have reportedly advised underrepresented students not to “overreach” by participating in Level IV or in taking honors classes at the middle school. Assuring students from historically marginalized populations feel a sense of belonging is an ongoing challenge of all advanced learning opportunities up to an including higher education. Although current efforts in FCPS to increase diversity in teaching and administrative staff were noted by some stakeholders, others believed not enough is being done to recruit these individuals to be teachers, administrators, and/or Central Selection Committee members. Minority principals and other principals who are also committed to diversity in teaching staff and who remained in their positions over several years are perceived as more effective in dealing with the issues noted.

34. The review team is aware, around the country, of central administrators’ general hostility toward the concept of advanced education, but even we were surprised at the depth of that feeling among some FCPS central administrators and some principals. Even the idea that advanced students may have academic needs that cannot be met in the regular classroom was met with scorn by many FCPS personnel working in the central administration. When asked for support for their strong beliefs, they vaguely referred to “research” that gifted education does not work, or that teachers can effectively differentiate for the entire range of performance and ability levels they may find in their classrooms or that the implementation of strategies to increase growth mindset would be sufficient to address the educational needs of gifted students. Several administrators shared that they would do away with all AAP services if not for “the political blowback,” as one person phrased it (they were not aware that having advanced programming is required in Virginia, providing additional evidence

41

that knowledge about advanced learning is in need of improvement among this group of FCPS educators).

The following example may be the exception that proves the rule: A parent of an ELL student praised their principal for serving as an advocate, sharing that they were told by the principal to “Go through me. I will help you with the process.” That principal also referred them to other parents who had gone through the process.

35. Articulation across levels of service appears uncommon. For example, most Center teachers reported rarely if ever having the opportunity to discuss AAP curriculum and efforts across levels of service AARTs, and vice-versa. Given the size of the division, such articulation can be challenging, but lack of communication across the levels of service is likely an impediment to the division’s equity efforts.

42

Recommendations 1. The goal of AAP is to identify and serve students with advanced academic needs that cannot be met

in the regular classroom. Effective efforts to achieve this goal must be based on ensuring the goal is embraced across all staff at all levels of FCPS. Current AAP identification systems are not designed to find students who would otherwise go under-challenged. Similarly, students who are served in Level IV have a wide range of needs, suggesting mismatch between needs and students remains. This all points to a lack of agreed-upon goals for AAP services throughout the division. As the division prepares the Local Plan for the Gifted for 2021-2026, clear goals for AAP services should be defined.

2. Over the next two years, the screening system should be revised. Changes should include: a. Discontinuation of NNAT universal administration. It assesses skills not directly related to

programming and does not appear to provide added value beyond the other universal assessments (for example, CogAT Composite scores and NNAT scores were correlated at .75). As was suggested by several stakeholders and verified by the literature on instrument validity, its use as a universally-administered assessment should be discontinued. Any universal testing is costly, but in this case, because of the overlap with CogAT, the inclusion of NNAT identifies relatively few additional students for the cost. CogAT non-verbal scores can be a source of data for consideration of Level I English-language learners.

b. Examine strategies to reduce the influence of the parent referral component of the identification process. The current identification system goes beyond Virginia requirements in ways that are not helpful to equity but certainly add administrative burden to the division (by allowing the inclusion of additional data from parents). The review team found considerable evidence that this mechanism is strongly biased in favor of some groups over others. The parent referral process should be revised to ensure that it does not act as an open pathway to certain groups but not others.

c. Reconsideration of use of the GBRS. There is currently very little, if any, evidence that the GBRS is a valid and reliable assessment tool for identifying advanced learners. It is currently a major factor in AAP service eligibility, despite very limited psychometric foundations. Further, the instrument is abused when teachers are allowed to “re-evaluate” a student based on parental requests. If teacher input is considered a vital part of the identification process, FCPS should identify an instrument that has been subjected to greater psychometric testing and use it judiciously with adequate (and consistent) training of teachers in interpretation of items according to the cultural context of students and conditions such as identified disability.

d. Examination of the initial universal screening cutoff score. The current cut scores for NNAT or CogAT are extremely high – only the top ~2% of students in the country would be expected to be further considered for identification and placement in Level IV Centers, even fewer from traditionally underrepresented groups. If the referral pathway was restricted, the universal screening cut scores could be lowered without overburdening the Central Selection Committee process. Furthermore, if local (i.e., building-level) norms were used to make placements, the cut scores would (and should) vary considerably among building.

e. Limitation of student work samples for the portfolios to school-based work. The current process allows students to include work samples, awards, or artwork in their portfolios and is another example of unequal data sources where students who have had greater numbers of opportunities will also have more awards and more artifacts to include. Ideally, all of the same data points would be available on all students who are considered for placement in gifted services. This can be accomplished by restricting the ability of families to include components

43

that stem directly from students’ ability to access a wider range of outside learning opportunities.

f. Reconsideration of the appeal process. Although the appeals process is not currently a major source of inequity (last year 587 out of 3,118 negative decisions were appealed), it could become a larger problem if the parent referral option is restricted or narrowed in the future. Ensuring that successful appeals are consistently, fairly, and infrequently granted will assure that the appeal process is not abused.

g. Reconsideration of allowing students to retake assessments. Although this option is only pursued by a small group of students each year, it is a likely source of bias in the identification process.

h. Review of central identification procedures. In particular, attention should be paid to (a) increasing the diversity of the Central Selection Committee, (b) training the Central Selection Committee to ensure equity in interpretation of data, inter-rater reliability, and as insurance against undue influence by a single, strong voice in the decision making process, and (c) other factors that have been noted above as likely exacerbating inequality.

3. Invest in frontloading opportunities. Improvements to the screening system will likely produce

equity benefits, but they will not achieve equity by themselves. FCPS should significantly improve its long-term approach to achieving equity in advanced learning by frontloading learning opportunities, early and often. a. Over the next three years, strengthen the Young Scholars program by implementing it fully and

consistently in all elementary schools, requiring every teacher in those schools to have professional development on the purpose, structure, and goals of the program as well as strategies for helping Young Scholars succeed. In addition, principals should be held accountable for the success of the program by including Young School program goals on their annual School Improvement Plan. Finally, Young Scholar program funding should be increased to augment the consistency and comprehensiveness of the program and to support the attendance of all Young Scholars at the summer programs. Narrowing opportunity gaps in this fashion (e.g., similar to Project EXCITE described above) are the most effective way to close identification gaps.

b. Over the next five years, expand Local Level IV services into every elementary and middle school with accompanying review and support for high level implementation. Identification for Local Level IV services should be committee decisions based on local/school-based norms to the extent possible, primarily to reinforce that every school has academically talented students, regardless of their current performance levels. 1. Comprehensive Local Level IV services across the division will require the use of local norms

in some cases to fill Local Level IV classes. 2. Provide the training and support for Local Level IV teachers in differentiation. Use of local

norms will require Local Level IV teachers to differentiate more effectively. It may also be possible to introduce some flexibility into the model and identify students who can attend Local Level IV classes part-time in one or two but not all academic areas if they demonstrate exceptional ability beyond those that would be served in the Level III environment. This already happens in some schools; their experiences could serve as models for others. Both of those new requirements are net positives and will help advance equity over time. i. A note of caution: Even principals who recommended creating Local Level IV programs

in each school – and there were many – noted that this will be a major cultural and instructional change in some schools. They emphasized that such a change would place a great deal of collective responsibility on school and division leaders (and create space issues for many); principals would need to ensure Level IV programs have qualified

44

teachers in the classrooms; central office staff will need to evaluate the programs on a regular basis; regional superintendents would need to hire and train principals with oversight of Local Level IV programs in mind and would be required to evaluate principals on those responsibilities; and central office leaders would need to support the development and maintenance of the Local Level IV centers through the recruitment of qualified teachers and professional development before placement. Given administrator attitudes toward and expertise in advanced education, the review team is not fully convinced this is feasible in the short-term (see next set of recommendations).

3. Create cluster groups for Level III students at every grade level. Grouping 5-7 students with advanced math and/or advanced reading per classroom is essential in ensuring teachers will use their skills to differentiate curriculum for these students beyond what the AART teacher can offer. Students with advanced math or reading skills do not have that advanced level of achievement for the limited time an AART has the instructional responsibility for that child’s learning.

4. Over the next five years, FCPS should commit to providing all division personnel with professional

development on the needs of advanced students and strategies for advanced education. Short-term priority should be placed on training administrators on the needs of academically advanced students, research-based strategies for promoting and supporting advanced achievement, and students who are twice-exceptional. Without strong administrator support, it is difficult to envision continued progress toward FCPS’ AAP equity goals. a. The division should ensure, over the next five years, that all teachers receive professional

development on advanced learning and learners, with emphasis on the difference between potential and performance, students who are twice exceptional, and cultural and resource issues that may result in underperformance of talented students. Given the important, if limited, role of teachers in the identification process, this professional development should include training on best practices in identification and controlling implicit bias. 1. Joint professional development for AART and Special Education teachers on students who

are twice exceptional may have a particular benefit such as, helping to create AAP-SPED partnerships within each school and prevent situations in which educators feel students should only receive one or the other set of services.

b. Over the next five years, FCPS should increase the number of AARTs across the division, with the goal of placing at least 1.0 FTE in each elementary school. Larger schools, especially Title I schools should be supported in the hiring of additional AART FTE (beyond 1.0). If Level IV services are offered in all middle schools, placing an AART in each school at that level would also be advisable. In addition, the AART’s time should not be included in the master schedule, and the AART job description should be focused on overseeing equity in AAP as it relates to that particular school (e.g., in the school’s referrals to the pool and in Levels II and III in the school).

c. The division should consider adding a specialist in the education of twice exceptional students, possibly shared between Advanced Academic Programs and Special Education, to provide additional expertise and guidance to parents.

d. Principal training should include success stories from schools that have created collaborative teams involving their AARTs, equity leads, and parent liaisons (which appears to maximize the efficiency and effectiveness of all three types of positions).

e. Over the next two years, the school board and superintendent should develop concrete and measurable criteria for holding regional superintendents and principals accountable for developing academic talent in every region and school. This is critically important, as some division leaders hold anti-excellence attitudes and advocate for practices that are not research-

45

supported – even in the face of evidence these beliefs and practices are likely worsening equity issues.

5. Change perceptions that the path to academic success only runs through the Thomas Jefferson High

School of Science and Technology. A major motivating factor for parent over-involvement in the AAP identification process is the perception that future academic (and life) success is primarily attainable via attendance at the division’s selective high school, and that learning in a Level IV center is highly predictive of admission to the high school. Many talented students attend other FCPS high schools and go on to achieve admission to prestigious universities and success in life. Sharing those stories more widely and enthusiastically may help counter the current perceptions of the selective high school as the only path for families wanting their talented child to be prepared for college and workforce success. Further, a school division the size of FCPS can support multiple selective high schools, lessening the parent focus on selection for the Thomas Jefferson High School of Science and Technology High School of Science and Technology. For example, each region could have a selective high school and/or selective high schools could be created with a range of disciplinary foci (e.g., medical science, government and international studies, performing and visual arts). Other large districts (e.g., Boston) have multiple selective high schools and have had success recently in diversifying their student populations.

Prioritization of Recommendations In this Recommendations section, the review team provided a series of detailed recommendations, distinguishing between those actions the division should implement with the next year or two and those that can be phased in over the course of the next three to five years. Because the above-mentioned recommendations are presented thematically, here we provide a prioritized summary of the recommendations:

Clarify the goals of AAP programming and levels of services within FCPS (Recommendation 1 above). Each level of AAP should be based on meeting particular student needs that cannot be met elsewhere. Cluster grouping and acceleration should be expanded to challenge a wider range of students and Local Level IV services should also be expanded to provide greater access to advanced learning at all buildings. This is a foundational action step on which all other activities will be based.

Strengthen the frontloading provided by the Young Scholars program (Recommendation 3) and coordinate AAP talent development efforts with other division- and community-wide efforts to mitigate the effects of poverty and inequality. The review team believes this to be the most important intervention if FCPS is serious about achieving equity in advanced programs. Given that this is a long-term intervention, efforts to improve Young Scholars should be put into place as soon as possible. Increase resource allocations of personnel (AARTs in Title 1 schools) and funding for summer programming as needed to create a more intensive intervention.

Make improvements to the screening process, including elimination of NNAT administration, a more restrictive policy toward appeals, limiting retesting to cases of demonstrated need, and rethinking the use of the GBRS, among several other steps (Recommendation 2). Many of these changes should result in cost savings or be cost neutral, making them relatively easy to implement in terms of resources. The review team views these identification improvements to be necessary but not sufficient for achieving equity in advanced programs.

46

Improve educator knowledge and skills regarding advanced learning and advanced learners (Recommendation 4). The review team recommends a rethinking of who receives such training and how that training is administered, with a priority being professional development for central administrators and principals. The next priority would be continuing and strengthening professional development of AART staff given the importance of AARTs to the success of advanced academic programs in each school. Finally, all classroom teachers in FCPS should have an understanding of the needs of diverse, advanced learners, including comprehensive knowledge of the Young Scholars program and how best to support participating students. Providing access, including provision of substitute teachers, should be a priority given the high quality professional development programming that has already been developed.

In a related vein, consider putting accountability and incentives in place for regional superintendents and principals regarding advanced learning (Recommendation 4). If the division emphasized that yearly learning growth is important – and being monitored – for all of its advanced students, efforts to strengthen AAP and the learning of a diverse group of advanced students would take on added urgency.

Consider adding a specialist in the education of students who are twice exceptional to provide additional expertise and guidance to parents (Recommendation 4). Given FCPS’ already impressive work in this area, creating a joint position shared between Advanced Academic Programs and Special Education is a logical next step in meeting the needs of students who are twice exceptional.

Increase the number of AARTs, with the goal of placing at least 1.0 FTE in each elementary school (Recommendation 4). The review team sees this as a longer-term goal for FCPS. However, the team notes that increasing accountability and incentives for advanced achievement would likely encourage more principals to find the resources in their budgets to make this happen sooner rather than later.

Start changing the narrative that success is only achieved through graduation from Thomas Jefferson High School of Science and Technology (Recommendation 5). This action step can take many forms; the review team recommends more energetic sharing of success stories from the division’s other high-quality high schools and creation of more selective high schools. Sharing success stories can occur starting immediately; creating more selective high schools is obviously a longer-term activity, especially in the current economic context. However, given the relatively long timeline for creating such schools, serious attention should be devoted to the possibilities in the next 1-2 years.

47

References

Abuhassàn, A., & Bates, T. C. (2015). Grit: Distinguishing effortful persistence from conscientiousness. Journal of Individual Differences, 36, 205–214.

Assouline, S. G., Ihrig, L. M., & Mahatmya, D. (2017). Closing the excellence gap: Investigation of an expanded talent search model for student selection into an extracurricular STEM program in rural middle schools. Gifted Child Quarterly, 61, 250–261. https://doi.org/10.1177/0016986217701833.

Briggs, C. J., Reis, S. M., & Sullivan, E. E. (2008). A national view of promising programs and practices for culturally, linguistically, and ethnically diverse gifted and talented students. Gifted Child Quarterly. https://doi.org/10.1177/0016986208316037

Brown, E. F., & Abernethy, S. H. (2009). Policy implications at the state and district level with RTI for gifted students. Gifted Child Today, 32(3), 52–57. https://doi.org/10.1177/107621750903200311

Burgoyne, A. P., Hambrick, D. Z., Moser, J. S., & Burt, S. A. (2018). Analysis of a mindset intervention. Journal of Research in Personality, 77, 21-30. https://doi.org/10.1016/j.jrp.2018.09.004

Burgoyne, A. P., Hambrick, D. Z., & Macnamara, B. N. (2020). How firm are the foundations of mind-set theory? The claims appear stronger than the evidence. Psychological Science, DOI: 0956797619897588.

Burnette, J. L., Hoyt, C. L., Russell, V. M., Lawson, B., Dweck, C. S., & Finkel, E. (2020). A growth mind-set intervention improves interest but not academic performance in the field of computer science. Social Psychological and Personality Science, 11(1), 107-116. https://doi.org/10.1177/1948550619841631

Card, D., & Giuliano, L. (2015). Can universal screening increase the representation of low income and minority students in gifted education? (No. 21519). Retrieved from http://www.nber.org/papers/w21519

Chmielewski, A. K., & Reardon, S. F. (2016). Patterns of cross-national variation in the association between income and academic achievement. AERA Open, 2(3), 1-18. doi:10.1177/2332858416649593

Clinkenbeard, P. R. (2012). Motivation and gifted students: Implications of theory and research. Psychology in the Schools, 49(7), 622-630.

Coalition for Psychology in Schools and Education. (2015). Top 20 principles from psychology for PreK–12 teaching and learning. American Psychological Association. Available at http://www.apa.org/ed/schools/cpse/top-twenty-principles.pdf

Cohen, G. L., Garcia, J., Apfel, N., & Master, A. (2006). Reducing the racial achievement gap: A social-psychological intervention. science, 313(5791), 1307-1310.

Cook, J. E., Purdie-Vaughns, V., Garcia, J., & Cohen, G. L. (2012). Chronic threat and contingent belonging: protective benefits of values affirmation on identity development. Journal of Personality and Social Psychology, 102(3), 479-496.

Cuba, M. J. (2020). Frontloading Academic Vocabulary for English Learners With Disabilities in an Integrated Classroom Setting. Intervention in School and Clinic, 55(4), 230–237. https://doi.org/10.1177/1053451219855736

Dixson, D. D., Keltner, D., Worrell, F. C., & Mello, Z. (2018). The magic of hope: Hope mediates the relationship between socioeconomic status and academic achievement. The Journal of Educational Research, 111(4), 507–515. https://doi.org/10.1080/00220671.2017.1302915

Dixson, D. Olszewski‐Kubilius, P., Subotnik,. R. F., & Worrell, F. C. (2020). Developing academic talent as a practicing school psychologist: From potential to expertise.

48

Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087.

Elhoweris, H., Mutua, K., Alsheikh, N., & Holloway, P. (2005). Effect of children’s ethnicity on teachers’ referral and recommendation decisions in gifted and talented programs. Remedial and Special Education, 26, 25–31. https://doi.org/10.1177/07419325050260010401

Finn, C. E., & Wright, B. L. (2015). Failing our brightest kids: The global challenge of educating high-ability students. Cambridge, MA: Harvard Education Press.

Flore, P. C., Mulder, J., & Wicherts, J. M. (2018). The influence of gender stereotype threat on mathematics test scores of Dutch high school students: a registered report, Comprehensive Results in Social Psychology, 3(2), 140-174, DOI: 10.1080/23743603.2018.1559647

Ford, D. Y., Grantham, T. C., & Whiting, G. W. (2008). Another look at the achievement gap: Learning from the experiences of gifted black students. Urban Education, 43, 216–239. doi:10.1177/0042085907312344

Gandhi, J., Watts, T. W., Masucci, M. D., Francis, D. V., & Darity, W. A. (2020). Isolation: An alternative to the “acting white” hypothesis in explaining Black under-enrollment in advanced courses. Journal of Economics, Race, and Policy. https://doi.org/10.1007/s41996-020-00051-4

Gandhi, J., & Raver, C. C. (2020). The effects of two mindset interventions on low-income students’ academic and psychological outcomes. Journal of Research on Educational Effectiveness, 1-29. https://doi.org/10.1080/19345747.2019.1711272

Grissom, J. A., & Redding, C. (2016). Discretion and Disproportionality: Explaining the Underrepresentation of High-Achieving Students of Color in Gifted Programs. AERA Open, 2(1), 2332858415622175. https://doi.org/10.1177/2332858415622175

Hamilton, R., McCoach, D. B., Tutwiler, M. S., Siegle, D., Gubbins, E. J., Callahan, C. M., Broderson, A. V., & Mun, R. U. (2017). Disentangling the roles on institutional and individual poverty in the identification of gifted students. Gifted Child Quarterly, 62, 6-24. doi:10.1177/0016986217738053

Hanover Research (2016, February). Examining the cost-effectiveness of AP exam fee subsidies for underrepresented students. Arlington, VA: Author. Retrieved May 15, 2017 from http://gssaweb.org/wp-content/uploads/2016/04/Examining-The-Cost-Effectiveness-Of-AP-Exam-Fee-Subsidies-For-Underrepresented-Students.pdf

Harradine, C. C., Coleman, M. R. B., & Winn, D. C. (2014). Recognizing academic potential in students of color: Findings of U-STARS~PLUS. Gifted Child Quarterly, 58, 24-34. doi:10.1177/0016986213506040

Head, K., Hammer, R. A., & Keesey, S. (2019). Twice Exceptional- Reaching Full Potential. Posters-at-the-Capitol. Retrieved from https://digitalcommons.murraystate.edu/postersatthecapitol/2020/WKU/5

Horn, C. V. (2015). Young scholars: A talent development model for finding and nurturing potential in underserved populations. Gifted Child Today, 38(1), 19-31. doi:10.1177/1076217514556532

Kettler, T., & Hurst, L. T. (2017). Advanced academic participation: A longitudinal analysis of ethnicity gaps in suburban schools. Journal for the Education of the Gifted, 40(1), 3–19. https://doi.org/10.1177/0162353216686217

Kids Count Data Center. (n.d.). Children in poverty by race and ethnicity. Retrieved from https://datacenter.kidscount.org/data/tables/44-children-in-poverty-by-race-and-ethnicity#detailed/1/any/false/871,870,573,869,36,868,867,133,38,35/10,11,9,12,1,185,13/324,323

Kyburg, R. M., Hertberg-Davis, H., & Callahan, C. M. (2007). Advanced Placement and International Baccalaureate: Optimal learning environments for talented minorities? Journal of Advanced Academics, 18, 172-215.

49

Lakin, J. M. (2018). Making the cut in gifted selection: Score combination rules and their impact on program diversity. Gifted Child Quarterly. https://doi.org/10.1177/0016986217752099

Lohman, D. F. (2005). Identifying academically talented minority students. Retrieved from https://nrcgt.uconn.edu/wp-content/uploads/sites/953/2015/04/rm05216.pdf

Mehta, J. (2015). The problem with grit. Education Week. Available at http://blogs.edweek.org/edweek/learning_deeply/2015/04/the_problem_with_grit.html

Menken, K. (2010). NCLB and English language learners: Challenges and consequences. Theory Into Practice, 49(2), 121–128. https://doi.org/10.1080/00405841003626619

Matthews, M. S., & Peters, S. J. (2018). Methods to increase the identification rate of students from traditionally underrepresented populations for gifted services. In S. I. Pfeiffer, E. Shaunessy-Dedrick, & M. Foley-Nicpon (Eds.), APA handbooks in psychology®. APA handbook of giftedness and talent (p. 317–331). American Psychological Association. https://doi.org/10.1037/0000038-021

McClain, M.-C., & Pfeiffer, S. (2012). Identification of gifted students in the United States today: A look at state definitions, policies, and practices. Journal of Applied School Psychology, 28(1), 59–88. https://doi.org/10.1080/15377903.2012.643757

McBee, M. T., Peters, S. J., & Waterman, C. (2014). Combining scores in multiple-criteria assessment systems: The impact of combination rule. Gifted Child Quarterly, 58(1), 69–89. https://doi.org/10.1177/0016986213513794

McBee, M. T., Peters, S. J., & Miller, E. M. (2016). The impact of the nomination stage on gifted program identification: A comprehensive psychometric analysis. Gifted Child Quarterly, 60(4), 258–278. https://doi.org/10.1177/0016986216656256

Moreau, D., Macnamara, B. N., & Hambrick, D. Z. (2019). Overstating the role of environmental factors in success: A cautionary note. Current Directions in Psychological Science, 28(1), 28-33. https://doi.org/10.1177/0963721418797300

National Center for Educational Statistics. (2016a). Number and percentage of public school students participating in English language learner (ELL) programs, by state: Selected years, fall 2004 through fall 2014. Retrieved from https://nces.ed.gov/programs/digest/d16/tables/dt16_204.20.asp

National Center for Educational Statistics. (2016b). Children 3 to 21 years old served under Individuals with Disabilities Education Act (IDEA), Part B, by type of disability: Selected years, 1976-77 through 2014-15. Retrieved from https://nces.ed.gov/programs/digest/d16/tables/dt16_204.30.asp

National Center for Education Statistics. (2017). Racial/ethnic enrollment in public schools. Retrieved from https://nces.ed.gov/programs/coe/indicator_cge.asp

Nguyen, H. H. D., & Ryan, A. M. (2008). Does stereotype threat affect test performance of minorities and women? A meta-analysis of experimental evidence. Journal of Applied Psychology, 93(6), 1314-1334.

Olszewski-Kubilius, P., & Corwith, S. (2018). Poverty, academic achievement, and giftedness: A literature review. Gifted Child Quarterly, 62(1), 37–55. https://doi.org/10.1177/0016986217738015

Olszewski-Kubilius, P., Steenbergen-Hu, S., Thomson, D., & Rosen, R. (2017). Minority achievement gaps in STEM: Findings of a longitudinal study of Project EXCITE. Gifted Child Quarterly, 61(1), 20–39. https://doi.org/10.1177/0016986216673449

Olszewski-Kubilius, P., & Steenbergen-Hu, S. (2017). Blending research-based practices and practice-embedded research: Project EXCITE closes achievement and excellence gaps for underrepresented gifted minority students. Gifted Child Quarterly, 61(3), 202–209. https://doi.org/10.1177/0016986217701836

50

Pennington, C. R., Heim, D., Levy, A. R., & Larkin, D. T. (2016). Twenty years of stereotype threat research: A review of psychological mediators. PLoS One, 11(1) : e0146487. Published online 2016 Jan 11. doi: 10.1371/journal.pone.0146487

Peters, S. J., & Engerrand, K. G. (2016). Equity and excellence: Proactive efforts in the identification of underrepresented students for gifted and talented services. Gifted Child Quarterly, 60(3), 159–171. https://doi.org/10.1177/0016986216643165

Peters, S. J., Rambo-Hernandez, K., Makel, M. C., Matthews, M. S., & Plucker, J. A. (2019). Effect of local norms on racial and ethnic representation in gifted education. AERA Open, 5(2), 2332858419848446. https://doi.org/10.1177/2332858419848446

Plucker, J. A. (1994). Introducing female scientists, mathematicians, and engineers into the curriculum: Location and evaluation of resources. Journal of Women and Minorities in Science and Engineering, 1(3), 209–220

Plucker, J. A., & Peters, S. J. (2020). Excellence gaps in education: Expanding opportunities for talented students. Harvard Education Press.

Plucker, J. A., Peters, S. J., & Schmalensee, S. (2017). Reducing excellence gaps: A research-based model. Gifted Child Today, 40(4), 245–250. https://doi.org/10.1177/1076217517723949

Plucker, J. A., Peters, S. J., & Schmalensee, S. (2017b). A model for eliminating excellence gaps. In J. Plucker, A. Rinn, M. Makel (Eds.), From giftedness to gifted Education: Reflecting theory in practice (pp. 319-335). Prufrock Press.

Powers, J. T., Cook, J. E., Purdie-Vaughns, V., Garcia, J., Apfel, N., & Cohen, G. L. (2016). Changing environments by changing individuals: The emergent effects of psychological intervention. Psychological Science, 27(2), 150-160.

Reardon, S. F., & Portilla, X. A. (2016). Recent trends in income, racial, and ethnic school readiness gaps at kindergarten entry. AERA Open, 2(3), 2332858416657343. https://doi.org/10.1177/2332858416657343

Renzulli, J. S., & Reis, S. M. (1985). The Schoolwide Enrichment Model: A comprehensive plan for Educational excellence. Mansfield Center, CT: Creative Learning Press.

Rimfeld, K., Kovas, Y., Dale, P. S., & Plomin, R. (2016). True grit and genetics: Predicting academic achievement from personality. Journal of Personality and Social Psychology, 111(5), 780-789. http://dx.doi.org/10.1037/pspp0000089

Ritchie, S. J., & Tucker-Drob, E. M. (2018). How much does education improve intelligence? A meta-analysis. Psychological Science, 29, 1358-1369 doi:10.1177/0956797618774253

Robinson, A., Adelson, J. L., Kidd, K. A., & Cunningham, C. M. (2018). A talent for tinkering: Developing talents in children from low-income households through engineering curriculum. Gifted Child Quarterly, 62(1), 130–144. https://doi.org/10.1177/0016986217738049

Schmader, T. (2002). Gender identification moderates stereotype threat effects on women's math performance. Journal of Experimental Social Psychology, 38(2), 194-201.

Sisk, V. F., Burgoyne, A. P., Sun, J., Butler, J. L., & Macnamara, B. N. (2018). To what extent and under which circumstances are growth mind-sets important to academic achievement? Two meta-analyses. Psychological science, 29(4), 549-571. https://doi.org/10.1177/0956797617739704

Spencer, S. J., Logel, C, & Davies, P. G. (2016). Stereotype threat. Annual Review of Psychology, 67(1), 415-437.

Spencer, S. J., Steele, C. M., & Quinn, D. M. (1999). Stereotype threat and women's math performance. Journal of experimental social psychology, 35(1), 4-28.

Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of personality and social psychology, 69(5), 797.

51

Subotnik, R. F., Olszewski-Kubilius, P., & Worrell, F. C. (2011). Rethinking giftedness and gifted education: A proposed direction forward based on psychological science. Psychological Science in the Public Interest, 12(1), 3–54. https://doi.org/10.1177/1529100611418056

VanTassel-Baska, J. (2013). The world of cross-cultural research: Insights for gifted education. Journal for the Education of the Gifted, 36(1), 6-18.

Usher, E. L., & Pajares, F. (2006). Sources of academic and self-regulatory efficacy beliefs of entering middle school students. Contemporary Educational Psychology, 31(2), 125–141. https://doi.org/10.1016/j.cedpsych.2005.03.002

Weiler, S., & Walker, S. (2009). Desegregating resegregation efforts: Providing all students opportunities to excel in advanced mathematics courses. Brigham Young University Education and Law Journal, 2009(2), 341–364. Retrieved from https://digitalcommons.law.byu.edu/elj/vol2009/iss2/5

Worrell, F. C., & Dixson, D. D. (2018). Retaining and recruiting underrepresented gifted students. In S. I. Pfeiffer (Ed.), Handbook of giftedness in children (2nd ed.). Springer.

Worrell, F. C., Subotnik, R. F., Olszewski-Kubilius, P., & Dixson, D. D. (2019). Gifted students. Annual Review of Psychology, 70(1), 551–576. https://doi.org/10.1146/annurev-psych-010418-102846

Yeager, D. S., & Walton, G. M. (2011). Social-psychological interventions in education: They’re not magic. Review of educational Research, 81(2), 267-301.

52

About the Review Team Expanded bios for the review team are included below. Additional information about the team’s experiences and expertise are available in the accepted project proposal. Dr. Carolyn M. Callahan, Commonwealth Professor of Education, University of Virginia Dr. Carolyn M. Callahan, Commonwealth Professor Emeritus at the University of Virginia. She developed the Master’s and Doctoral Programs in Gifted Education, taught classes in gifted education, and developed the Summer and Saturday Enrichment Programs at UVA. For her work at the University in teaching and service she received the She has been recognized as Outstanding Professor of the Commonwealth of Virginia. Dr. Callahan has been principal investigator on multiple projects of the National Research Center on the Gifted and Talented (NRC/GT) funded by the U.S. Department of Education, including projects investigating the identification of gifted and talented students, evaluation of gifted programs, identification of gifted students from under-represented populations, investigation of curricular and instructional interventions for gifted students, and study of schools of mathematics, science and technology. She is currently serving as the principal investigator of the National Center for Research on Gifted Education (also funded by the Institute of Education Sciences of the U.S. Department of Education which has been investigating the efficacy of strategies for identification of under-represented groups of gifted students and the effectiveness of select programming strategies (fulltime programming and part-time programming in mathematics and language arts). She has also conducted two evaluations of the Javits grant program funded by the U.S. Department of Education and she has been principal investigator on six Javits grants. Each of those grants has focused on under-represented populations, and they have ranged from the creating and implementation of curriculum for under-represented gifted students at the elementary school level, to study of Advanced Placement programming including on-line delivery systems to development of identification procedures and curriculum for gifted students in high poverty rural schools. Her research has focused on evaluation. She co-authored the most widely cited and recognized guide to evaluation of gifted programs. Her research in the area of identification of gifted students included working on the team that developed and validated one of the most widely used teacher rating scales in the field (Scales for Rating the Behavioral Characteristics of Gifted Students) and she has more than two dozen articles on her research in the area of under-represented gifted students. Her research in the area of curriculum development and implementation was published in the most esteemed journal in education, American Educational Research Journal. She has conducted evaluations of more than 50state and local level gifted programs large and small. Dr. Callahan’s research has been recognized by the National Association for Gifted Children Distinguished Scholar Award. She served as President of the National Association for Gifted Children and the Association for the Gifted and as Editor of Gifted Child Quarterly and Journal for the Education of the Gifted.. Dr. Callahan has published over 250 articles and 60 book chapters on topics in gifted education. Dante Dixson, Assistant Professor of School and Educational Psychology, Michigan State University Dante D. Dixson received his Bachelor’s degree (Honors) in psychology, Master’s degree in education, and Ph.D. in School Psychology from the University of California, Berkeley. Currently, he serves as a

53

tenure-track faculty member at Michigan State University within the School and Education Psychology Programs and is certified for the practice of psychology both inside and outside of schools. The author or co/author of over 50 scholarly articles, research presentations, and book chapters, Dr. Dixson is a recipient of the Cota Robles Fellowship (from UC Berkeley) and an African American Success Foundation Research Grant. His areas of expertise include the role of hope in the educational and psychological functioning of children and adolescents, psychosocial precursors of achievement, at-risk youth, the achievement gap, academic talent development/gifted education, and the translation of psychological research findings into school-based practice. Dr. Dixson currently serves on the editorial boards of Gifted Child Quarterly, Journal for the Education of the Gifted, and Journal of Black Psychology. In addition, he is a board member for the Roeper Institute, Boys Hope Girls Hope-Detroit, and the Michigan Association of Gifted Children. Dr. Dixson is a member of several professional associations, including the American Educational Research Association, the American Psychological Association, the National Association for Gifted Children, and the Association for Psychological Science. Previously, Dr. Dixson served as the Managing Editor of Review of Educational Research from 2013 through 2016, an editorial board member for New School Psychology Bulletin from 2015 to 2017, and as an intervention consultant for Detroit Public Schools from 2016 to 2017. Scott Peters, Professor of Assessment & Research Methodology, University of Wisconsin-Whitewater Scott J. Peters is a Professor of Assessment and Research Methodology and the Richard and Veronica Telfer Endowed Faculty Fellow of Education at the University of Wisconsin – Whitewater. He received his Ph.D. from Purdue University specializing in gifted and talented education and applied research methodology. His research work focuses on educational assessment and data use, gifted and talented student identification, equity within advanced educational opportunities, and educational policy. He has published in the Australian Educational Researcher, AERA Open, Teaching for High Potential, Gifted Child Quarterly, Journal of Advanced Academics, Journal for the Education of the Gifted, Gifted and Talented International, Gifted Children, Journal of Career and Technical Education Research, Educational Leadership, Education Week, and Pedagogies. He is the first author of Beyond Gifted Education: Designing and Implementing Advanced Academic Programs (2013) and Designing Gifted Education Programs and Services: From Purpose to Implementation (2017), both from Prufrock Press, and the co-author (along with Jonathan Plucker) of Excellence Gaps in Education: Expanding Opportunities for Talented Students (2016), published by Harvard Education Press. Dr. Peters is the recipient of the Feldhusen Doctoral Fellowship in Gifted Education, the NAGC Research an Evaluation Network Dissertation Award, the NAGC Doctoral Student of the Year Award, the NAGC Early Scholar Award, the NAGC Paper of the Year Award, the NAGC Book of the Year Award, and the UW-Whitewater Innovation and Outstanding Research Awards. He currently serves as the Association Editor for NAGC and has served as the Program Chair of the AERA Research on Giftedness, Creativity, and Talent SIG, on the Board of Directors of the Wisconsin Association for Talented and Gifted, and as the secretary of the National Association for Gifted Children Research and Evaluation Network.

54

Jonathan Plucker, Julian C. Stanley Professor, Johns Hopkins University Prof. Jonathan Plucker is the Julian C. Stanley Endowed Professor of Talent Development at Johns Hopkins University, where he works in the Center for Talented Youth and School of Education. Previously, he was Raymond Neag Endowed Professor of Education at the University of Connecticut and Professor of Educational Psychology and Cognitive Science at Indiana University, where he was the founding director of the Center for Evaluation and Education Policy (CEEP). Under Prof. Plucker’s leadership, CEEP grew over a decade into one of the biggest university-based education research centers in the country, with projects in all 50 states and several other countries around the globe. Topics of CEEP research and policy projects ranged from early reading to science education, from special education to gifted education, and from state-level policy assistance to governors and legislators to research support for members of the U.S. Congress. He graduated with a B.S. in chemistry education and M.A. in educational psychology from the University of Connecticut, then after briefly teaching at an elementary school in New York, received his Ph.D. in educational psychology from the University of Virginia. His research examines education policy and talent development, with over 300 publications to his credit and over $40 million in external funding to support his work. His recent books include Excellence Gaps in Education with Scott Peters (Harvard Ed Press) and Creativity and Innovation (Prufrock Press), both of which have received the National Association for Gifted Children Book Award, From Giftedness to Gifted Education with Anne Rinn and Matthew Makel (Prufrock Press), and Toward a More Perfect Psychology with Matthew Makel (American Psychological Association). Prof. Plucker is a Fellow of the American Psychological Association, Association for Psychological Sciences, American Educational Research Association, and the American Association for the Advancement of Science. He was the recipient of the 2012 Arnheim Award for Outstanding Achievement in the Psychology of Creativity from the American Psychological Association and 2013 Distinguished Scholar Award from the National Association for Gifted Children. He is a past-president of the Society for the Psychology of Aesthetics, Creativity, and the Arts, and he began his term as president of the National Association for Gifted Children in September 2019.

55

Appendix A Highlights of Relevant Sections of CHAPTER 40: REGULATIONS GOVERNING EDUCATIONAL SERVICES FOR

GIFTED STUDENTS (§ 22.1-16 of the Code of Virginia) 8VAC20-40-40. Screening, referral, identification, and service

Each school division shall establish uniform procedures for screening, referring, identifying, and serving students in kindergarten through twelfth grade who are gifted in general intellectual or specific academic aptitude

These uniform procedures shall include a screening process that requires instructional personnel to review, at a minimum, current assessment data on each kindergarten through twelfth-grade student annually. Some data used in the screening process may be incorporated into multiple criteria reviewed by the designated identification and placement committee to determine eligibility, but those data shall not replace norm-referenced aptitude test data.

These uniform procedures shall permit referrals from parents or legal guardians, teachers, professionals, students, peers, self, or others.

An identification and placement committee shall review pertinent information, records, and other performance evidence for referred students. The committee shall consider input from a professional who knows the child. The committee shall include classroom teachers, assessment specialists, gifted program staff, school administrators, or others with credentials or experience in gifted education. The committee shall (i) review data from multiple sources selected and used consistently within the division to assess students' aptitudes in the areas of giftedness the school division serves, (ii) determine whether a student is eligible for the division's services, and (iii) determine which of the school division's service options match the learning needs of the eligible student. The committee may review valid and reliable data administered by another division for a transfer student who has been identified previously.

o Identification of students for the gifted education program shall be based on multiple criteria established by the school division and designed to seek out those students with superior aptitudes, including students for whom accurate identification may be affected because they are economically disadvantaged, have limited English proficiency, or have a disability. Data shall include scores from valid and reliable instruments that assess students' potential for advanced achievement, as well as instruments that assess demonstrated advanced skills, conceptual knowledge, and problem-solving aptitudes.

o Valid and reliable data for each referred student shall be examined by the building-level or division-level identification and placement committee. The committee shall determine the eligibility of each referred student for the school division's gifted education services. Students who are found eligible by the identification and placement committee shall be offered service options with appropriately differentiated curriculum and instruction by the school division.

o The identification process used by each school division must ensure that no single criterion is used to determine a student's eligibility. The identification process shall include at least three measures from the following categories:

Assessment of appropriate student products, performance, or portfolio; Record of observation of in-classroom behavior; Appropriate rating scales, checklists, or questionnaires; Individual interview; Individually administered or group-administered, nationally norm-referenced

aptitude or achievement tests; Record of previous accomplishments (such as awards, honors, grades, etc.); or

56

Additional valid and reliable measures or procedures. If a program is designed to address general intellectual aptitude, an individually

administered or group-administered, nationally norm-referenced aptitude test shall be included as one of the three measures used in the school division's identification procedure.

Each school division shall adopt a review procedure for students whose cases are appealed. This procedure shall involve a committee, the majority of whose members did not serve on the initial identification and placement committee, and shall inform parents or legal guardians, in writing, of the appeal process. Requests filed by parents or legal guardians to appeal any action of the identification and placement committee shall be filed within 10 instructional days of receipt of notification of the action by the division. The process shall include an opportunity to meet with an administrator to discuss the decision

57

Appendix B Sample focus group and interview protocols: Principals

General Instructions: Begin each interview with a brief statement of purpose

We are here to gather information about the AAP program in the school division. Your input will be combined with that of many others as we consider the current status of the program and make recommendations to the administration. You and anyone you might name in this interview will remain anonymous to the degree that is possible.

I would like you to begin by just telling me about your perceptions of the services offered to gifted students in the Fairfax County Public Schools. 1. What do you see as the strengths of the program? The weaknesses of the program? 2. How is the curriculum offered to the gifted something other children ‐‐other than those identified ‐‐

could not do, would not do and should not do? a. In your school, how does the program differ across levels offered?

3. Would you describe the accomplishments of the gifted students that would not be possible without

the program?

4. Does it (the identification process) miss other students who might benefit from the instruction in the program? Who? Why do you think so?

5. In what ways are you involved with the identification of gifted students? administration of the

program? a. Could you describe for me how you see the process working in your school? What is the

process? b. How are persons selected to serve on the identification team in your school?

6. What policies and procedures are in place to ensure equity in identification across all racial, ethnic

and socio‐economic groups? a. Are these policies and procedures sufficient to ensure equity? b. What are the barriers to greater equity? Are there some groups for whom the equity is an

issue? c. What else could be done to increase equity? d. Are there students missed by the current process who could have benefited from services?

What could be done to ensure we find those students?

7. What do you see as the relative strengths and weaknesses of the Center vs Local options for Level IV services?

8. What contribution does the Young Scholars Program make, if any, to increasing equity in the

Advanced Academic Program? At all levels?

58

Sample Protocol: Central Selection Committee I will begin with some global questions to give me a perspective on the goals of gifted services offered in the Fairfax County Publics Schools.

1. In your opinions, what are appropriate goals for services offered to gifted students in the school division? That is, what do you expect should be achieved by students in the gifted program that could not be achieved if the gifted program did not exist?

a. What do you believe is the primary goal of Level IV services across the school division? b. Do you see those goals as the same or different across the different levels of

programming (centers, pullout option, in-class option)? Perceptions about gifted program and services

2. What do you believe are the prevailing perceptions throughout the school division of this committee and its work?

3. What do you believe are the prevailing perceptions of Level IV services throughout the Division?

a. Do you believe that the perceptions of the other levels of services offered to gifted

students (differentiation and pull-out) are the same or different?

Perceptions about gifted identification

4. Describe the process of identification of students for Level IV services in Fairfax County Public Schools and the way this committee fits into that process.

a. What are the strengths of the process? b. What are the weaknesses of the process? c. How do you perceive the identification process for Level IV services is similar and

different than the identification process for other levels of services? d. Are these similarities and differences justifiable? Why or why not?

Perceptions about equity within gifted services

5. What policies and procedures are in place to ensure equity in identification for Level IV services

across all racial, ethnic and socio‐economic groups?

a. Are these policies and procedures sufficient to ensure equity? b. What are the barriers to greater equity? Are there some groups for whom equity is still

an issue? c. What else can be done to increase equity? d. Are there students missed by the current process who could have benefited from

services? What could be done to ensure we find those students?

6. Do you believe there are any equity issues relative to the delivery of Level IV services (e.g., curriculum, ways students are instructed, etc.)?

59

a. What are your thoughts on what has been done to ensure equity within Level IV services?

b. Is there a sub‐group or individual that is responsible for overseeing diversity within Level IV services?

c. How are equitable practices monitored within Level IV services? d. Is this process working? e. How can it be improved?

7. What role, if any, has the Young Scholars Program played in increasing equity in Level IV

services?

60

Appendix C Parent Survey

3/2/2020 Fairfax County Public Schools Advanced Academic Programs Parent Survey

https://docs.google.com/forms/d/1OJBYFa_lTeugI0G_p4tE6ne5sFTGGXTv85jIRmGNnzs/edit 1/4

Fairfax County Public Schools Advanced AcademicPrograms Parent SurveyThis survey is being conducted by a team of external researchers to recommend ways to increase the diversity of students served by FCPS Level IV Advanced Academic Programs (AAP). Although FCPS offers a continuum of AAP services to meet student needs, this survey focuses solely on Level IV. Level IV services are full-time services in the four core content areas (Mathematics, Science, Language Arts, and Social Studies). Level IV services include "local" Level IV services (offered in the student's neighborhood school) as well as centralized Level IV "centers" (sites that draw from multiple feeder schools).

In addition to analyzing FCPS data, reviewing policies and identification procedures, and holding focus groups with families, staff, and community members, we have designed this survey to gather feedback from parents and guardians about their experiences with and perceptions of AAP Level IV services in FCPS.

Information about AAP in FCPS:https://www.fcps.edu/academics/elementary-school-academics-k-6/advanced-academics

At the end of the survey are several optional demographic questions. We are including these questions in order to see if any particular groups or families are not aware of certain AAP services or how they can be accessed.

* Required

1. Which of the following roles applies to you? *Check all that apply.

Parent or guardian of a child enrolled in an FCPS AAP Level IV center.

Parent or guardian of a child enrolled in an FCPS AAP local Level IV program.

Parent or guardian of a child served by an FCPS AAP service other than Level IV (Level II or III).

Parent or guardian of a child who has never been served by FCPS AAP.

Other:

2. Which of the following applies to you? Check all that apply. *Check all that apply.

I have an elementary student in FCPS.

I have a middle school student in FCPS.

Other:

3. Do you have a student who has been screened for level IV services? *Check all that apply.

Yes

No

Unsure

3/2/2020 Fairfax County Public Schools Advanced Academic Programs Parent Survey

https://docs.google.com/forms/d/1OJBYFa_lTeugI0G_p4tE6ne5sFTGGXTv85jIRmGNnzs/edit 2/4

4. I understand how students are selected to attend one of the Level IV centers.(https://tinyurl.com/yzeechoz) *Mark only one oval.

Strongly Agree

Agree

Disagree

Strongly Disagree

5. I understand how students are selected to attend a local Level IV program. *Mark only one oval.

Strongly Agree

Agree

Disagree

Strongly Disagree

6. The Level IV AAP selection process does a good job of choosing students to attend the LevelIV centers. (List of centers: https://tinyurl.com/yfdjxg2k) *Mark only one oval.

Strongly Agree

Agree

Disagree

Strongly Disagree

7. One pathway to Level IV services involves a parent referral. Were you aware of this referralprocess? *Mark only one oval.

I am aware of the referral process and understand how it works.

I know there is a referral process but am not sure how it works.

I did not know there was a referral process.

8. I am aware that I can appeal any decision about Level IV service eligibility. *Mark only one oval.

Yes

No

9. The Level IV AAP selection process fails to identify students who would benefit from attending(either at the local level or a center). *Mark only one oval.

Strongly Agree

Agree

Disagree

Strongly Disagree

Unsure

3/2/2020 Fairfax County Public Schools Advanced Academic Programs Parent Survey

https://docs.google.com/forms/d/1OJBYFa_lTeugI0G_p4tE6ne5sFTGGXTv85jIRmGNnzs/edit 3/4

10. The Level IV selection process is biased against certain student / family groups. *Mark only one oval.

Strongly Agree

Agree

Disagree

Strongly Disagree

Unsure

11. FCPS should continue to offer the Level IV centers as one of its AAP options. *Mark only one oval.

Yes

No

Unsure

12. In your opinion, what could be done to improve the Level IV AAP selection process?

13. In your opinion, what could be done to diversify the student body served by Level IV AAPservices?

14. Based on what you know about the Level IV AAP selection process, what are the greatestbarriers for families to overcome in order for students to be selected?

3/2/2020 Fairfax County Public Schools Advanced Academic Programs Parent Survey

https://docs.google.com/forms/d/1OJBYFa_lTeugI0G_p4tE6ne5sFTGGXTv85jIRmGNnzs/edit 4/4

Powered by

15. (Optional) With which racial / ethnic groups do you identify? Check all that apply.Check all that apply.

African American / Black

Asian American

European American / White

Latino/a or Hispanic

Native American

Multi-Racial

Other:

16. (Optional) Is your family eligible for free or reduced-price meals at school?Mark only one oval.

Yes

No

Unsure

17. (Optional) Do you have a child who is served both by special education services and FCPSAdvanced Academic Programming (i.e. twice exceptional)?Mark only one oval.

Yes

No

18. (Optional) Is English the primary language spoken in your home?Mark only one oval.

Yes

No

Thank you. Please click "Submit" to record your answers. If youhave any additional comments or questions for the researchteam, please submit them to Dr. Scott Peters atpeterss@uww.edu.

65

Appendix D NNAT, CogAT, and GBRS Score Descriptive Statistics for Universally Screened Students

Total GBRS

White Black Hispanic Native Asian Two+ Hawaiian /

Pac

Valid 5405 1351 3505 45 2582 925 24

Missing 0 0 0 0 0 0 0

Mean 10.480 9.715 9.442 9.867 11.033 10.813 9.875

Std.

Deviation 2.753 2.437 2.247 2.436 2.912 2.959 2.610

Minimum 8.000 8.000 8.000 8.000 8.000 8.000 8.000

Maximum 16.000 16.000 16.000 16.000 16.000 16.000 16.000

NNAT

White Black Hispanic Native Asian Two+ Hawaiian / Pac

Valid 4988 1205 3324 44 2386 848 19

Missing 417 146 181 1 196 77 5

Mean 107.471 97.583 98.264 101.977 113.583 108.892 102.368

Std. Deviation 13.376 14.538 13.731 21.539 17.583 14.447 15.710

Minimum 40.000 40.000 40.000 40.000 48.000 40.000 76.000

Maximum 160.000 142.000 145.000 158.000 160.000 160.000 133.000

CogAT NV

White Black Hispanic Native Asian Two+ Hawaiian / Pac

Valid 5141 1235 3157 42 2411 870 22

Missing 264 116 348 3 171 55 2

Mean 110.750 101.195 100.811 103.000 114.805 112.069 106.909

Std. Deviation 13.602 13.227 13.647 15.460 14.208 14.263 13.180

Minimum 60.000 52.000 50.000 64.000 58.000 63.000 74.000

Maximum 142.000 140.000 141.000 137.000 144.000 143.000 136.000

CogAT composite

White Black Hispanic Native Asian Two+ Hawaiian / Pac

Valid 5120 1222 3133 42 2404 868 22

Missing 285 129 372 3 178 57 2

Mean 112.128 101.241 99.585 104.119 115.676 113.452 105.636

Std. Deviation 14.270 13.827 13.770 17.127 15.479 15.188 11.770

Minimum 54.000 59.000 55.000 63.000 55.000 64.000 77.000

Maximum 153.000 155.000 144.000 143.000 158.000 151.000 128.000

66

Appendix E NNAT, CogAT, and GBRS Score Descriptive Statistics for Level IV-eligible Students

Descriptive Statistics

NNAT

White Black Hispanic Native Asian Multi Haw

Valid 827 144 257 5 638 199 1

Missing 53 10 14 0 39 11 0

Mean 118.266 111.056 112.708 133.800 129.019 120.789 102.000

Std. Deviation 11.932 11.169 10.301 25.004 15.047 11.056 N/A

Minimum 70.000 85.000 85.000 96.000 93.000 90.000 102.000

Maximum 160.000 142.000 143.000 158.000 160.000 160.000 102.000

Descriptive Statistics

Cog AT Verbal

White Black Hispanic Native Asian Multi Haw

Valid 876 154 270 5 676 210 1

Missing 4 0 1 0 1 0 0

Mean 122.120 114.266 114.085 115.400 120.587 122.714 117.000

Std. Deviation 9.722 10.130 10.552 3.912 10.192 9.291 N/A

Minimum 93.000 90.000 89.000 110.000 92.000 99.000 117.000

Maximum 151.000 143.000 139.000 121.000 154.000 150.000 117.000

Descriptive Statistics

CogAT Quantitative

White Black Hispanic Native Asian Multi Haw

Valid 876 154 270 5 676 210 1

Missing 4 0 1 0 1 0 0

Mean 127.299 119.838 118.930 135.600 130.953 128.248 120.000

Std. Deviation 10.029 11.701 11.874 12.779 9.228 8.835 N/A

Minimum 90.000 89.000 87.000 114.000 100.000 103.000 120.000

Maximum 150.000 149.000 144.000 148.000 152.000 148.000 120.000

Descriptive Statistics

CogAT Non-verbal

White Black Hispanic Native Asian Multi Haw

Valid 875 154 270 5 676 210 1

Missing 5 0 1 0 1 0 0

Mean 124.206 116.013 118.344 125.800 127.303 125.105 122.000

Std. Deviation 8.814 10.716 10.148 11.987 8.380 8.324 N/A

Minimum 89.000 89.000 93.000 110.000 90.000 104.000 122.000

Maximum 142.000 140.000 141.000 137.000 144.000 141.000 122.000

67

Descriptive Statistics

Total GBRS

White Black Hispanic Native Asian Multi Haw

Valid 880 154 271 5 677 210 1

Missing 0 0 0 0 0 0 0

Mean 14.327 14.208 14.137 14.200 14.292 14.452 12.000

Std. Deviation 1.833 1.740 1.846 2.168 1.872 1.774 N/A

Minimum 8.000 8.000 8.000 11.000 8.000 8.000 12.000

Maximum 16.000 16.000 16.000 16.000 16.000 16.000 12.000

68

Appendix F Parent Survey Results Summary

The parent survey was designed with the goal of providing additional context and insight on equity within Level IV AAP services. In addition to select-type responses described below, we collected parent / guardian comments to help us better understand specific responses or policies. These comments are not summarized below; rather we integrated them into our major findings throughout the report.

The summary presented below is based on the same survey administered in eight total languages. However, it’s important to note than English results dominated (~94% of responses). For this reason, we highlight where non-English responses varied rather than try and summarize the results altogether, which would dilute any non-English responses. Most often we compare summary responses to the English survey with those from the Spanish survey due to sample size.

Respondents: Roughly 90% of the parents who responded to the survey were parents or guardians of students served by a Level IV Center, a local Level IV program, or a local Level III AAP. Level IV Center parents / guardians were the plurality (~41%). The vast majority were parents or guardians of elementary FCPS students (~83%).

Survey respondents were not representative of the low-income demographics of the district. For example. 95% of English and 73% of Spanish respondents were not eligible for free or reduced-price meals.

12% of English and 24% of Spanish respondents reported children being served by special education services.

90% of English respondents reported English as the language spoken at home, suggesting even non-native English speakers may have responded to the English survey, thereby explaining some of the low non-English response rates.

Awareness of policies: ~78% of English respondents stated that they understood the selection process for Level IV Centers – only slightly less (~75%) understood the Local Level IV selection process. These values were similar across all languages of the survey.

85% of respondents to the English survey were aware of the parental referral process and 64% (of all respondents) felt they understood it. However, this dropped off with Spanish respondents – 47% of whom said they were not aware of this option at all.

80% of English survey respondents were aware that they could appeal any eligibility decision for Level IV services. This was 54% for Spanish respondents.

Judgement of policies: Overall, parents and guardians felt the Level IV selection process does a good job of identifying students (~62%; ~82% for Spanish respondents). 27% of parents strongly disagreed that the Level IV selection process does a good job.

69

When asked in a different question, ~40% of parents or guardians agreed that the selection process misses students who would benefit from Level IV services. This number rose to 60% for Spanish respondents. There seems to be a clear sense that the selection process works somewhat well, but misses some students and could use some improvements.

Just over a quarter of respondents felt the selection process was biased against certain groups (~36% for Spanish respondents). In reviewing parent comments, it was clear this bias came from all perspectives. Some felt there was bias against non-native English speakers, others had concerns about bias against Asian families, and some pointed out the logistics required and how those could create unintentional barriers for families.

Level IV Centers: The majority of parents felt the Centers should continue to exist as one service option for advanced learners (~61%). ~22% were unsure and ~17% disagreed. Respondents to the non-English versions of the survey were more in favor of the Centers continuing (e.g., Spanish ~83%, Korean ~76%, Chinese 78%).