what factors most threaten the validity of test performance?

Unless otherwise indicated, information contained in this packet is Copyright © Samuel O. Ortiz, Ph.D. May not be reproduced without permission.

Using the Culture-Language Interpretive Matrix (C-LIM v2.0) as part of a nondiscriminatory approach

to psychoeducational evaluation of SLD.

Webinar for Education Service Centers

October 15, 2013

Samuel O. Ortiz, Ph.D. St. John’s University

Critical Examination of Current and Typical Methods of Assessment with English Learners:

What works, what doesn’t, and why.


What Factors Most Threaten the Validity of Test Performance?

“Most studies compare the performance of students from different ethnic groups…

rather than ELL and non-ELL children within those ethnic groups….A major

difficulty with all of these studies is that the category Hispanic includes students

from diverse cultural backgrounds with markedly different English-language

skills….This reinforces the need to separate the influences of ethnicity and ELL

status on observed score differences.”

Lohman, Korb & Lakin, 2008, p. 276-278.

Developmental Language Proficiency – Not Language Dominance

Acculturative Knowledge Acquisition – Not Race or Ethnicity

“When a child’s general background experiences differ from those of the

children on whom a test was standardized, then the use of the norms of that

test as an index for evaluating that child’s current performance or for

predicting future performances may be inappropriate.”

Salvia & Ysseldyke, 1991


Nondiscriminatory Assessment: Processes and Procedures

IX. REDUCE BIAS IN TRADITIONAL TESTING PRACTICES

Exactly how is evidence-based, nondiscriminatory assessment conducted?

• Modified Methods of Evaluation

• Modified and altered testing

• Nonverbal Methods of Evaluation

• Language reduced assessment

• Native Language Evaluation

• Bilingual assessment

• English Language Evaluation

• Assessment of bilinguals



ISSUES IN MODIFIED METHODS OF EVALUATION

Modified and Altered Assessment:

• “testing the limits:” alteration or modification of test items or content, mediating task concepts prior to administration, repeating instructions, accepting responses in either language, and eliminating or modifying time constraints may all help the examinee perform better, but violates standardization

• “translator/interpreter:” use of a translator/interpreter for administration helps overcome the language barrier but also undermines score validity, even when the interpreter is highly trained and experienced; tests are not usually normed in this manner

• alterations or modifications are perhaps most useful in deriving qualitative information—observing behavior, evaluating learning propensity, evaluating developmental capabilities, analyzing errors, etc.

• a recommended procedure would be to administer tests in a standardized manner first, which will potentially allow for later interpretation, and then consider any modifications or alterations that will further inform the referral questions



ISSUES IN NONVERBAL METHODS OF EVALUATION

Language Reduced Assessment:

• “nonverbal testing:” use of language-reduced ( or ‘nonverbal’) tests are helpful in overcoming the language obstacle, however:

• it is impossible to administer a test without some type of communication occurring between examinee and examiner, this is the purpose of gestures/pantomime

• some tests remain very culturally embedded—they do not become culture-free simply because language is not required for responding

• construct underrepresentation is common, especially on tests that measure fluid reasoning (Gf), and when viewed within the context of CHC theory, some batteries measure a narrower range of broad cognitive abilities/processes, particularly those related to verbal academic skills such as reading and writing (e.g., Ga and Gc) and mathematics (Gq)

• all nonverbal tests are subject to the same problems with norms and cultural content as verbal tests—that is, they do not control for differences in acculturation and language proficiency which may still affect performance, albeit less than with verbal tests

• Language reduced tests are helpful in evaluation of diverse individuals and may provide better estimates of true functioning in certain areas, but they are not a whole or completely satisfactory solution with respect to fairness and provide no information about dysfunction in the most common areas of referral (i.e., reading and writing) or in mathematics



ISSUES IN NATIVE LANGUAGE EVALUATION

Bilingual Assessment:

• refers to the assessment of bilinguals in a bilingual manner by a bilingual psychologist

• the bilingual psychologist is in a position to conduct assessment activities in a manner (i.e. bilingually) that is not available to the monolingual psychologist even with the aid of interpreter

• bilingual assessment is a relatively new research tradition with little empirical support to guide appropriate activities or upon which to base standards of practice

• there are no truly “bilingual” tests or assessment protocols and not much is yet known about the performance of bilinguals on monolingual tests administered in the primary language

• the relative lack of competent, trained, and qualified bilingual psychologists limits the chances that students will be evaluated in this way, especially in languages other than Spanish



ISSUES IN ENGLISH LANGUAGE EVALUATION

Assessment of Bilinguals:

• refers to the assessment of bilinguals in a monolingual manner by a monolingual psychologist

• extensive research exists regarding performance of bilinguals on tests given in English

• goal is to reduce bias to maximum extent possible even through the use of tests given in English

• testing in English allows for the use of systematic methods based on established literature and research for collecting and interpreting data in a nondiscriminatory manner (e.g., CHC Culture-Language Interpretive Matrix)

• does not require that the evaluator speak the language of the child but does require competency, training and knowledge, in nondiscriminatory assessment including the manner in which cultural and linguistic factors affect test performance


Nondiscriminatory Assessment: Evaluation Methods and Evidence-based Practice

Evaluation Method

Norm sample representative

of bilingual development

Measures full range of ability

constructs

Does not require

bilingual evaluator

Adheres to the test’s

standardized protocol

Substantial research base on

bilingual performance

Modified or Altered Assessment

Reduced-language Assessment

Native-Language Assessment

English-Language Assessment

Addressing issues of fairness with respect to norm sample representation is an issue of validity and dependent on a sufficient research base.


A Recommended Best Practice Approach for Using Tests with ELLs

Step 1. Assessment of Bilinguals – validate test scores (difference vs. disorder)

• Select or create an appropriate battery that is comprehensive and responds to the needs of the referral concerns, irrespective of language differences

• Administer all tests in standardized manner in English only, no modifications

• Score tests and plot them for analysis via the C-LIM

• If analysis indicates expected range and pattern of decline, evaluation ends, no disability is likely

• If analysis does not indicate expected range or pattern of decline, apply XBA (or other) interpretive methods to determine specific areas of weakness and difficulty and continue to Step 2

Step 2. Bilingual Assessment – validate disorder (cross-language confirmation)

• Review prior results and create a select set of tests related to the areas where the suspected weaknesses or difficulties were noted

• Select tests that are as parallel as possible to the original tests using one of 3 methods:1. Native language test administered in the native language (e.g., WJ III/Bateria III or WISC-IV/WISC-IV Spanish)

2. Native language test administered via assistance of a trained interpreter

3. Informally translated test administered via assistance of a trained interpreter

• Administer all tests in whatever manner necessary to ensure full comprehension including use of any modifications and alterations necessary to reduce barriers to performance

• Observe and document approach to tasks, errors in responding, and behavior during testing

• Analyze data both quantitatively and qualitatively to evaluate areas of weakness or difficulty

• If areas of weakness do not match areas of weakness from Step 1 analyses, disability NOT likely

• If areas of weakness match areas of weakness from Step 1 analyses, disability is likely, except for Gc

• If testing of Gc in native language reveals better functioning than in English, use native language Gc score


WECHSLER INTELLIEGENCE SCALE FOR CHILDREN-IV

Verbal Comprehension Index 76 Perceptual Reasoning Index 88 Working Memory Index 79

Vocabulary 5 Block Design 10 Letter-Number Seq. 6

Comprehension 7 Matrix Reasoning 8 Digit Span 7

Similarities 5 Picture Concepts 5

Processing Speed Index 94

Coding 9

Symbol Search 9

WECHSLER INDIVIDUAL ACHIEVEMENT TEST-III

Basic Reading 94 Reading Comprehension 76 Written Expression 92

Word Reading 92 Reading Comprehension 76 Spelling 100

Pseudoword Decoding 98 Oral Reading Fluency 80 Sentence Composition 86

Essay Composition 93

WOODCOCK JOHNSON-III TESTS OF COGNITIVE ABILITY

Auditory Processing 92 LT Storage/Retrieval 67

Auditory Attention 92 Visual Auditory Learning 69

Sound Blending 94 Retrieval Fluency 74

WISC-IV/WJ III/WIAT-III XBA DATA FOR MAGDALENA

Using the XBA Software in SLD Identification: A Case Study



Most important consideration is determination of student’s degree of “difference” regarding language

development and acculturative acquisition



0

Not supportive of expected pattern of decline, suggests cultural and

linguistic factors are not primary



Results not fully explainable by cultural and linguistic influences alone--other factor is

present and affecting performance



Possible Gsm problem

Possible Gf problem

Gc is in shaded range on C-LIM graph—indicates average development as

compared to other English Learners and is NOT likely

a problem area.

Possible Gc problem?


Statement 2. Evaluations of Suspected Learning Disability - Valid Results

The following sample validity statement may be used in cases where a clear declining pattern is NOT evident, that is, there is no primary effect of culture and language thus the results ARE valid and there may be a disability.

Because the student is not a native English speaker, it is necessary to establish the validity of the results obtained from testing to ensure that they are accurate estimates of ability or knowledge and not the manifestation of cultural or linguistic differences. To this end, a systematic evaluation of the possible effects of lack of acculturation and limited English proficiency was carried out via use of the Culture-Language Interpretive Matrix (C-LIM). A careful review of the student’s test data as entered into the C-LIM does not appear to reveal a pattern of decline that is typical of or within the range that would be expected of other individuals with similar cultural and linguistic backgrounds. The overall pattern of test performance does not decline systematically and suggests that test performance was not due primarily to the influence of cultural and linguistic factors. Although such influences remain contributory factors, they can not account for the resulting pattern of performance in its entirety and are, therefore, not believed to be the main or only reason for the reported learning difficulties. In addition, other extraneous factors that might account for the observed pattern (for example, lack of motivation, fatigue, incorrect administration/scoring, emotional/behavioral problems) have been excluded. This indicates that the test results can be considered valid, interpretable, and are likely to be good estimates of the student’s actual ability or knowledge with the exception of Gc, which must be evaluated only against other ELLs due to the fact that it is a direct measure of cultural knowledge and language proficiency.

In summary, the observed pattern of the student's test results is not consistent with performance that is typical of non-disabled, culturally and linguistically diverse individuals who are of average ability or higher. Therefore, it can be reasonably concluded that the data evaluated with the C-LIM are likely valid and that, if supported by additional data, the student’s test performance may be attributed primarily to the presence of a learning disability.

(*Note: a typical description of the data that support the presence of LD should follow here at this point in the report.)



General Steps for Using the XBA DMIA in SLD Evaluations:

1. Select the tab that corresponds to the core battery used in the evaluation and enter all available data. Data from supplemental batteries should be entered on the corresponding battery first for analysis and may be entered afterwards on core battery tab.

2. Review analyses of cohesion and follow up recommendations. Where cohesion is listed as “need clinical judgment,” it will be necessary to determine if supplemental testing is needed to form a cohesive cluster.

3. Select subtests that require follow up and check appropriate boxes to transfer scores to CHC Analyzer for evaluation of composite. If available, manually enter supplemental subtest data for inclusion in evaluation of domain composites.

4. Evaluate broad ability domains and determine if data result in formation of composite scores for use in PSW-A. If not, consider supplemental testing.




DMIA recommends no follow up on any academic composites, however, further evaluation of Total Reading composite is

recommended



Grw-R – Reading decoding (SS=94) represents a narrow ability area of strength but Reading Comprehension and Fluency (SS=76) is a definite area of academic weakness

Grw-W – Writing ability falls within normal limits and does not appear to be an area of academic weakness



DMIA recommends follow up for PRI—not surprising since PRI is a Gf/Gv mixed index


The XBA DMIA v2.0 provides guidance regarding the cohesion of composite and cluster scores and whether follow up is necessary to bolster the measurement reliability and construct validity of the ability domain.

Scores can be easily transferred to the CHC Analyzer to evaluate the composition and interpretability of two to four scores within any given CHC broad ability domain.


Use of clinical clusters indicates follow up recommendation for Gf and need for additional Gv

subtest to create broad ability composite



Use of clinical clusters indicates need for follow up (no composite

formed) for Gf and need for additional Gv subtest to create

broad ability composite


For Gf – The WISC-IV Word Reasoning subtest is another measure of Induction and also loads strongly on Gc. Thus, the WJ III Analysis-Synthesis subtest makes better sense because it measures a different narrow (RG) ability and loads only one broad ability (Gf).

For Gv – The WISC-IV Picture Completion subtest loads strongly on Gc, not just Gv. Thus, the WJ III Picture Recognition subtest makes better sense because it measures a different narrow (MV) ability and loads on only one broad ability (Gv).



WECHSLER INTELLIEGENCE SCALE FOR CHILDREN-IV

Verbal Comprehension Index 76 Perceptual Reasoning Index 88 Working Memory Index 79

Vocabulary 5(75) Block Design 10(100) Letter-Number Seq. 6(75)

Comprehension 7(85) Matrix Reasoning 8(90) Digit Span 7(85)

Similarities 5(75) Picture Concepts 5(75)

Processing Speed Index 94

Coding 9(95)

Symbol Search 9(95)

WECHSLER INDIVIDUAL ACHIEVEMENT TEST-III

Basic Reading 94 Reading Comprehension 76 Written Expression 92

Word Reading 92 Reading Comprehension 76 Spelling 100

Pseudoword Decoding 98 Oral Reading Fluency 80 Sentence Composition 86

Essay Composition 93

WOODCOCK JOHNSON-III TESTS OF COGNITIVE ABILITY

Auditory Processing 92 LT Storage/Retrieval 67 Other

Auditory Attention 92 Visual Auditory Learning 69 Picture Recognition 97

Sound Blending 94 Retrieval Fluency 74 Analysis-Synthesis 90

WISC-IV, WIAT-III, AND WJ III DATA FOR MAGDALENA




Supplemental WJ III tests given for purposes of follow up now included in matrix



Scores still not entirely explained by cultural and linguistic factors—indicates results

continue to be valid and may be interpreted meaningfully


Follow up for Gv indicates that performance resulted in a cohesive and valid cross-battery composite for Gv of 98 indicating that Visual Processing abilities are well within the average range.

Follow up for Gf indicates that performance on the WISC-IV Picture Concepts subtest appears to be an anomaly (perhaps due to the Gc content) and that a valid cross-battery composite for Gf can be formed between the other two subtests. This suggests that Fluid Reasoning ability is within the average range.




Use of CHC Tab provides summary of broad ability composites for both cognitive and academic tests.

Data are now sufficient for the purposes of the PSW-A to evaluate SLD.

Use norm-based scores where available, use XBA composites as necessary.

Scores must be transferred to PSW-A by hand (sorry!).


General Steps for Using the XBA PSW-A in SLD Evaluations:

1. Enter required information, particularly grade. Move to g-Value data entry tab and enter a composite score for each of the seven broad ability domains and indicate if score represents “sufficient” ability for each domain.

2. Review g-Value and note whether additional information is needed to determine if it is indicative of average overall ability or not. If yes, continue to PSW-A data entry tab.

3. Enter required information on PSW-A data entry tab and make appropriate selections or accept default values. If an alternative to IA-e is desired, enter it here. For repeated analyses, select stricter probability level for analyses.

4. Review PSW-A tab. If data are missing, return to appropriate tab and enter data. When all data are entered, review pattern to determine if all criteria are met for indication of SLD.

5. Review recommendations and interpretive statements on PSW-A summary tab. Print summary pages as desired.



Guidelines for Using PSW-A with ELLs

Step 1

Step 2

Step 3

Step 1: Is the aggregate value for the “high/high” cell in the C-LIM within the selected difference band?

(touches or exceeds the shaded area corresponding to the selected degree of difference range)

Enter actual/obtained Gc composite score as deemed appropriate from prior specified guidelines

Student is unlikely to be SLD and other possibilities should be considered, e.g., slow learner or intellectual disability as may be

supported by additional evidence.

YES

Indicate Gc ability as “sufficient,” and conduct

PSW analyses

Was the IA-e calculated by the PSW-A (SS > 85)?

Indicate Gc ability as “insufficient,” and conduct

PSW analyses

NO

NO

YES

Step 2. Enter alternative IA-e score in PSW-A (e.g., nonverbal IQ/Index/Composite) and re-run PSW-A analyses

Did the PSW-A Summary indicate a pattern consistent

with SLD?

Student is likely SLD, no further analyses necessary

NO

YES

Did the PSW-A indicate a pattern consistent with SLD?YES

NO

Step 3. Enter alternative Gc score that reflects minimum level of “average” ability, i.e., SS=90 and re-run analyses using IA-e


NO

YESDid the PSW-A Summary

indicate a pattern consistent with SLD?

NO

YES



Use of normative meaning (i.e., indicating that score is in the “not sufficient” range) for Gc will

inappropriately attenuate g-Value for ELLs.



Indicating Gc as “insufficient” for

ELLs may result in a g-Value that will not permit further evaluation of SLD

and unfairly suggests a lack of

average overall ability


Nondiscriminatory Interpretation of Test Scores: A Case Study

Because Gc is, by definition, comprised of cultural knowledge and language development, the influence of cultural and linguistic differences cannot be entirely separated from tests which are designed to measure culture and language. Thus, Gc scores for ELLs, even when determined to be valid, remain at risk for inequitable interpretation and evaluation.

Much like academic tests of manifest skills, Gc scores do reflect the examinee’s current level of English language proficiency and acculturative knowledge. However, they do so as compared to native English speakers, not to other ELLs. This is discriminatory and comparison of Gc performance using a test’s actual norms remains unfair when assigning meaning to the value. It is necessary instead to ensure that both the magnitude and the interpretive “meaning“ assigned to the obtained value is done in the least biased manner possible to maintain equity.

For example, interpretation of a Gc score of 76 for an ELL should be deemed “sufficient” because it falls within the expected range on the C-LIM when compared to other ELLs. Likewise, the magnitude of a Gc score of 76 is in the average range as compared to other ELLs as seen in the C-LIM. But such a score would become “deficient” relative to the norm sample where average scores are equal to 100. Thus, it may be necessary to use an alternative value to ensure that ELLs are not unfairly regarded as having either deficient Gc ability or significantly lower overall cognitive ability—conditions that may simultaneously decrease identification of SLD and increase suspicion of ID.

The Gc caveat for English Language Learners


Nondiscriminatory Interpretation of Test Scores: A Case Study

To address these issues in as fair and equitable a manner as possible when using the PSW-A with ELLs, specific guidelines have been developed. These guidelines:

• prevent the use of random, multiple analyses which would affect the rarity level in the PSW-A,

• maintain the nature of the discrepancy comparisons consistent with theory and meaning of the composites,

• provide a conservative and systematic mechanism for addressing fairness issues, and

• limit the need for adjustments to a small and unique set of conditions.

The actual, obtained Gc score, regardless of magnitude or sufficiency, should always be reported, albeit with appropriate nondiscriminatory assignment of meaning, and used for the purposes of instructional planning and educational intervention.

The Gc caveat for English Language Learners


Procedural Steps for Nondiscriminatory Evaluation of SLD with PSW-A: A declining pattern must NOT be evident in the C-LIM indicating no primary

(only contributory) effect of culture and language; The results have been deemed VALID via the C-LIM and may potentially

support the presence of a disability;

STEP 1: Enter the obtained standard score value for Gc and:a) If the aggregate score in the "high/high" cell in the C-LIM does not touch and is

below the shaded range in the graph, select "No" for Gc to indicate it is "not sufficient," or:

b) If the aggregate score in the "high/high" cell in the C-LIM touches or exceeds the shaded range in the graph, select "Yes" for Gc to indicate that it is "sufficient;“

c) If the IA-e is calculated by the PSW-A (was SS > 85) and subsequent analysis with the PSW-A resulted in a pattern consistent with SLD, no further analyses are necessary; student is likely to be SLD.

d) If the IA-e is not calculated by the PSW-A (was SS < 85) and prevents further SLD analysis, move on to Step 2.

Nondiscriminatory Interpretation of Test Scores: A Case StudyData Entry Guidelines for Using PSW-A with English Learners



Step 1

Step 2

Step 3






YES


PSW analyses



PSW analyses

NO

NO

YES



with SLD?


NO

YES


NO



NO



NO

YES



Gc performance is slightly higher than the average score/range when compared to other

English learners



Use of actual SS with nondiscriminatory meaning provides less biased and fair interpretation of ability in area of Gc



The g-Value now reflects true and

equitable estimate of overall cognitive ability and permits further evaluation

of SLD.


Using the XBA Software in SLD Identification: A Case Study – Scenario 1

However, for ELLs, even when Gc is

determined to be “sufficient,” the IA-e may not be

calculated because it

remains below the minimum value of 85.

In this case, it would be

necessary to proceed directly to Step 2 since

no further analysis of SLD can take place.



But in most cases, when Gc is determined to

be “sufficient” and the actual value is used, the PSW-A will

be able to calculate the IA-e which permits continuation of SLD evaluation.

Most significant cognitive weakness

entered first

Related academic weakness

entered here


In this case, data are not consistent with SLD pattern. However, this may be

because the actual Gc score (SS=76) is attenuating the IA-e (SS=86). Further

analysis should be conducted via Step 2 guidelines using an alternative IA-e

score.

If SLD was found at this point, no further analysis is necessary.

Pattern of strengths and weaknesses not consistent with SLD




Step 1

Step 2

Step 3






YES


PSW analyses



PSW analyses

NO

NO

YES



with SLD?


NO

YES


NO



NO



NO

YES


Procedural Steps for Nondiscriminatory Evaluation of SLD with PSW-A: A declining pattern is NOT be evident in the C-LIM indicating no primary (only

contributory) effect of culture and language; The results have been deemed VALID via the C-LIM and may potentially support the

presence of a disability; The IA-e was not calculated in Step 1 by the PSW-A (was SS < 85) and prevented further

SLD analysis; STEP 2: Enter an alternative index or composite score (e.g., nonverbal) that best reflects

overall ability, is SS > 85, and that is not affected by or relies on Gc ability and:a) If subsequent analysis with the PSW-A resulted in a pattern consistent with SLD, no further

analyses are necessary; student is likely SLD. b) If subsequent analysis with the PSW-A did NOT result in a pattern consistent with SLD, and

there is other evidence to support it, move on to Step 3.




Step 2: Use an alternative score

that may be less affected by

or doesn’t include Gc

ability (e.g., a nonverbal index or composite).

Entry of an alternate score

will enable program to

conduct calculations and analyses even

when IA-e is not calculated.

Minimum value for alternative

score is SS>85


In this case, data are consistent with SLD pattern. Use of fairer estimate of overall ability

demonstrates differences necessary to establish SLD. No further analyses required.




Step 2: If the alternative score is within the low average range, it may still provide

an underestimate of

true ability, particularly

because the highest g-loaded ability (i.e., Gc) is not factored

in.

Minimum value for alternative

score is SS>85


In this case, data are not consistent with SLD pattern possibly because the

removal of Gc takes away the most important ability related to cognitive and

academic functioning and does not provide a fair estimate of overall general

ability. Further analysis should be conducted via Step 3 guidelines using

an alternative Gc value.

If SLD was found at this point, no further analysis is necessary.

Pattern of strengths and weaknesses still not consistent with SLD




Step 1

Step 2

Step 3






YES


PSW analyses



PSW analyses

NO

NO

YES



with SLD?


NO

YES


NO



NO



NO

YES


Procedural Steps for Nondiscriminatory Evaluation of SLD with PSW-A: A declining pattern is NOT be evident in the C-LIM indicating no primary (only

contributory) effect of culture and language; The results have been deemed VALID via the C-LIM and may potentially support the

presence of a disability; The IA-e was not calculated in Step 1 by the PSW-A (was SS < 85) and prevented further

SLD analysis; Use of an alternative score (e.g., nonverbal index) resulted in a pattern of strengths and

weaknesses that were not consistent with SLD;STEP 3: Enter an alternative, less biased score for Gc that reflects equitable meaning

regarding relative performance and indicates minimum level of average ability (e.g., SS > 90), remove alternative nonverbal score from Step 2 (i.e., allow program to calculate the IA-e) and:

a) If the IA-e was not calculated by the PSW-A (was SS < 85) and prevented further SLD analysis, student is unlikely to be SLD.

b) If the IA-e was calculated by the PSW-A (was SS > 85) and subsequent analysis with the PSW-A resulted in a pattern consistent with SLD, student is likely SLD.

c) If the IA-e was calculated by the PSW-A (was SS > 85) and subsequent analysis with the PSW-A did NOT result in a pattern consistent with SLD, student is unlikely to be SLD.




Step 3: Enter an alternative value for Gc that corresponds to the minimum score

necessary for establishing average or better ability (i.e., SS=90).

Conduct further analyses with this value but note that its use is limited to the PSW-A only

in accordance with these guidelines and that the actual composite or index score for

Gc should be used for evaluation of instructional intervention and current levels

of performance.



Again, the g-Value is not affected by the magnitude of the standard score since it is based on “sufficiency” not on the

magnitude of the ability scores.



Notice that the IA-e is now less attenuated and

actually falls within the

average range so that it

appears in green in the program and indicates

average or better overall

cognitive ability.


Using the XBA Software in SLD Identification: A Case StudyFinal analysis of data via Step 3 indicates full

consistency with SLD pattern. Use of guidelines to ensure fair and unbiased assignment of meaning to

obtained values helps demonstrate differences necessary to establish SLD that might have been masked due to inherently attenuated Gc score.



PSW-A Summary shows solid support for SLD without any equivocation. Failure to follow steps for use of PSW-A with ELLs could lead to a decrease in the likelihood of finding true SLD as well as increase

in likelihood of misidentifying student as “slow learner” or intellectually impaired.


Statement 2a. Evaluations of Suspected Learning Disability – Gc Caveat

The following additional wording may be used in cases where: 1) a declining pattern is NOT evident indicating no primary (only contributory) effect of culture and language; 2) the results ARE deemed valid and may potentially support the presence of a disability; 3) initial analysis with the PSW-A using the actual Gc score resulted in a pattern NOT CONSISTENT with SLD; and 4) use of the PSW-A guidelines for ELLs produced a pattern that is CONSISTENT with SLD.

…This indicates that the test results can be considered valid, interpretable, and are likely to be good estimates of the student’s actual ability or knowledge with the exception of Gc, which must be evaluated only against other ELLs due to the fact that it is a direct measure of cultural knowledge and language proficiency. In this regard, initial evaluation of SLD with the PSW-A using the actual obtained Gc score resulted in an unfair estimate of overall cognitive ability that inequitably decreased the difference between the student’s strengths and weaknesses and masked the presence of SLD. To prevent biased evaluation, systematic steps were taken to ensure that the analysis was not subject to the use of inappropriate or discriminatory values or classification including one or all of the following: indication of Gc as “sufficient” if the score was comparable to other English learners, use of an alternative, nonverbal ability score in lieu of the IA-e, and entry of a value for Gc (SS=90) that accurately portrays the correct “average” magnitude for the true level of ability in this domain. Use of these procedures permitted nondiscriminatory analysis and resulted in a pattern of strengths and weaknesses consistent with the required conceptual and quantitative criterion necessary to establish SLD.

In summary, the observed pattern of the student's test results is not consistent with performance that is typical of non-disabled, culturally and linguistically diverse individuals who are of average ability or higher. Therefore, it can be reasonably concluded that the data evaluated with the C-LIM are likely valid and that, if supported by additional data, the student’s test performance may be attributed primarily to the presence of a learning disability.

(*Note: a typical description of the data that support the presence of LD should follow here at this point in the report.)



The Culture-Language Test Classifications and Interpretive Matrix: Caveats and Conclusions

Used in conjunction with other information relevant to appropriate bilingual, cross-cultural, nondiscriminatory assessment including…

- level of acculturation- language proficiency- socio-economic status- academic history- familial history- developmental data- work samples- curriculum based data- intervention results, etc.

…the C-LTC and C-LIM can be of practical value in helping establish credible and defensible validity for test data, thereby decreasing the potential for biased and discriminatory interpretation. Taken together with other assessment data, the C-LTC and C-LIM assist practitioners in answering the most basic question in assessment:

“Are the student’s observed learning problems due primarily to cultural or linguistic differences or disorder?”


XBA - Cross-Battery Assessment Resources

BOOKS:

Flanagan, D. P., Ortiz, S.O. & Alfonso, V.C. (2013). Essentials of Cross-Battery Assessment, Third Edition. New York: Wiley & Sons, Inc.

Flanagan, D.P. & Ortiz, S.O. (2012). Essentials of Learning Disability Identification. New York: Wiley & Sons, Inc.

Flanagan, D. P., Ortiz, S.O. & Alfonso, V.C. (2007). Essentials of Cross-Battery Assessment, Second Edition. New York: Wiley & Sons, Inc.

Flanagan, D.P., Ortiz, S.O., Alfonso, V., & Mascolo, J. (2006). The Achievement Test Desk Reference (ATDR): A guide to Learning Disability Assessment, 2nd Edition. New York: Wiley.

CHC Cross-Battery Online http://www.crossbattery.com/

ONLINE:

what factors most threaten the validity of test performance?

Documents

nondiscriminatory assessment

language barrier

altered assessment

modified methods of

typical methods of assessment

copyright samuel

validity of test performance

performance of students