running head: rating scale qualitykdbrad2/aera2005ratingscalequalitypaper.pdf · human sciences...
TRANSCRIPT
Rating Scale Quality 1
Running Head: RATING SCALE QUALITY
Quality Control in Survey Design: Evaluating a Rating Scale of Educators’ Attitudes
Toward Differentiated Compensation
Shannon O. Sampson
Kelly D. Bradley
University of Kentucky
Rating Scale Quality 2
Abstract
Researchers too often overlook the question of measurement; yet, the accuracy of the data
is only as good as the measure. For this reason, it is essential that measurement instruments be
assessed for quality. This study utilizes the Rasch model to assess the quality of a survey
instrument designed to measure attitudes of administrators and teachers about a differentiated
teacher compensation program piloted in Kentucky. While the instrument is found to be
sufficient for its purpose, this study provides suggestions for improvement, including
recommendations to enhance and develop items identified as misfitting. The method illustrated
in assessing the measurement tool will benefit the survey research community as well as the
general methodologist.
An Argument for Rasch Analysis 3
INTRODUCTION
Surveys are the most common example of self-reported data collection and continue to be
one of the most popular research methodologies for graduate studies and published papers in
education (Babbie, 1992; Gay, 1981). Even so, the development of instruments to assess
affective domain constructs has been a problematic area within the field of education (Aiken,
1996). Bond and Fox (2001) write, “Operationalizing and then measuring variables are two of
the necessary first steps in the empirical research process. Statistical analysis, as a tool for
investigating relations among the measures, then follows. Thus, interpretation of analyses can
only be as good as the quality of the measures.” The quality of the instrument used in the
measurement process must play a fundamental role in the analysis of the data collected from it.
It is important to begin at the level of measurement and to identify weaknesses that may limit the
reliability and validity of the measures made with the survey instrument. After the efficiency
and effectiveness of the measurement instrument have been addressed, only then should
statistical analysis of the data take place .
Some psychometricians and behavioral statisticians treat survey data as if the mere
assignment of numerical values to objects suffices as scientific measurement (Michell, 1997).
When researchers develop a group of items intended to assess a construct, administer the items to
a sample of respondents, and sum the ratings, certain assumptions are put in place:
Each item contributes equally to the measure of that construct, implying all items are of equal
importance.
Each item is measured on the same interval scale.
Respondents have appropriately interpreted the directions, all items are written clearly, and
the items tap the same construct, creating a single dimension.
An Argument for Rasch Analysis 4
In actuality, these assumptions are unstable, and often problematic, in survey research methods
(Bond & Fox, 2001, Sampson & Bradley, 2003). For example, in practice the scale is actually
ordinal, so categories are not necessarily spaced equally.
This study utilizes the Rasch model to assess the quality of the measurement instrument,
here a selected response survey, and the structure of the rating scale. Findings will benefit the
developers of the survey in ensuring the instrument is functioning as intended and will give
insight into possible revisions for future rounds of data collection. In addition, a methodological
framework for educational researchers developing survey instruments and analyzing rating scale
data is offered.
BACKGROUND
Rasch versus Classical Test Theory Approach
Researchers often utilize the classical test theory model in analyzing the rating scale data
produced via the selected-response survey. As noted in Smith (2000), the classical test theory
model, sometimes called the true score model, has deficiencies. This approach requires complete
records to make comparisons of items on the survey. Even if this is attained, the issue of sample-
dependence between estimates of an item’s difficulty to endorse and a respondent’s willingness
to endorse surface. This is problematic since it makes the estimates for the items dependent on
the rater-severity of the respondents in the sample. Moreover, the estimates of item difficulty
cannot be directly compared unless the estimates come from the same sample or assumptions are
made about the comparability of the different samples. Another concern with the classical test
theory approach is that only a single standard error of measurement is produced for the
composite of the ratings or scores, making it inadequate and potentially misleading.
An Argument for Rasch Analysis 5
In this study, a methodological framework is presented to address the concerns related to
the traditional classical test theory approach presented above. The quality of the instrument used
to collect the data is first assessed; then, the collected data are analyzed and interpreted in a
mathematically sound manner. Thus, the study presents a method for ensuring and/or
establishing the quality of the survey instrument.
The Rasch model, introduced by Georg Rasch (1960), addresses many of the weaknesses
of the classical test theory approach. It starts at the level of measurement to provide diagnostic
information on the quality of the measurement tool. In addition, it yields a more comprehensive
and informative picture of the construct under measurement as well as the respondents on that
measure. Specific to rating scale data, the Rasch model allows for the connection of
observations of respondents and items in a way that indicates the occurrence of a certain
response as probability rather than certainty and maintains order in that the probability of
providing a certain response defines an order of respondents and items. These circumstances
create the probabilistic version of the scalogram, which indicates a person endorsing a more
extreme statement should also endorse all less extreme statements, and an easy-to-endorse item
is always expected to be rated higher by any respondent (Wright and Masters, 1982). Applying
the Rasch model allows researchers to identify where possible misinterpretation occurs in the
instrument and which items do not appear to measure the construct of interest. Information
about the structure of the rating scale and the degree to which each item contributes to the
construct is also produced. The model provides a mathematically sound alternative to traditional
approaches of survey data analysis.
In contrast to classical test theory, parameters in the Rasch model are neither sample nor
test dependent so that missing data are not problematic. Furthermore, Rasch measurement
An Argument for Rasch Analysis 6
produces standard error estimates for each discrete raw score, allowing for one reliability
coefficient to be calculated for the instrument and another for the respondents. It is also possible
to combine any person’s estimated measure with any item’s estimated measure to produce
expected response value because respondents and items are measured on the same metric.
Applications of the Rasch model also provide estimates for persons and items that are freed from
the sampling distribution of the sample employed, meaning there is no dependence on the
particulars of the questionnaire nor of the sample being measured (Smith, E., 2000; Wright,
1997; Wright and Masters, 1982). It is important that quantitative methodologist within the
human sciences discontinue the practice of analyzing raw data or counts and instead analyze
measures (Bond and Fox, 2001).
The analysis of measures is central to the Rasch model. This study applies Rasch
techniques to a survey designed to measure the attitudes of teachers, mentor teacher-achievement
coaches, superintendents, and principals toward differentiated compensation programs, defined
as a range of incentives that are added on to present compensation. Such compensations include
salary bonuses for teaching in critical shortage areas; financial support for seeking advanced
degrees; or participation in voluntary career advancement opportunities. By attempting to
measure the variable, it is understood that although respondents have many significant and
distinct characteristics, only one characteristic can be meaningfully rated at a time.
Evaluating the Differentiated Teacher Compensation Program
The Kentucky Department of Education (KDE) provided support to ten school districts to
plan and pilot a differentiated teacher compensation program to determine policy implications
for (1) recruiting and retaining teachers in critical shortage areas; (2) reducing the number of
emergency certified teachers; (3) providing incentives for teachers to serve in difficult
An Argument for Rasch Analysis 7
assignments and hard-to-fill positions; (4) providing voluntary career advancement
opportunities; and (5) rewarding teachers who increase their knowledge and skills. An essential
component of the program is to ascertain the effectiveness of the program over a two-year period
(2003-2005). KDE appointed the College of Education at the University of Kentucky (UK) to
evaluate the Differentiated Compensation Pilot Site program 1. A team of UK faculty
constructed a selected response pencil-and-paper survey instrument as the essential measure,
making it a key tool in the evaluation procedure (see complete instrument in appendix).
Following the administration of the survey, a Rasch measurement approach was applied
to the response data set to assess the stability and quality of the measures. Assumptions specific
to the Rasch model guided the process of determining the quality of the set of measures related
to the differentiated compensation survey. Specifically, the analysis responded to the questions:
Does the survey measure a single variable?; and Does each person’s response pattern indicate
that he/she was responding in an acceptably predictable way given the expected hierarchy of
responses? (Wright and Stone, 1979). The results produce measures that adequately represent
the intended construct with a replicable and meaningful set of measures. Findings have an
immediate impact to the survey development team, while the methodology offers a framework
for others, especially those in the social and behavioral sciences.
1 The Principal Investigator in the study was Lars Björk, College of Education,
University of Kentucky
An Argument for Rasch Analysis 8
METHOD Participants
Surveys were distributed to districts across Kentucky that had participated in the
differentiated compensation pilot. It was a census sample administered to four groups:
superintendents (n = 10), principals (n = 63), teachers (n = 438), and mentor teacher-achievement
coaches (n = 60). The survey was not identical due to the differences by group; however each
form of the survey was analogous. Because of the nature and distribution format of the surveys,
the number of surveys distributed was approximately equivalent to the number of surveys
returned. Thus, representativeness of the sample on the population is not a concern.
Apparatus
The selected response 34-item pencil-and-paper survey, along with the subsequent data
collection tools were developed and conducted by the group of researchers at the University of
Kentucky. The items (see complete survey in the appendix) included topics ranging from the
effects of differentiated compensation on factors such as standardized test scores, staff relations,
teacher morale and teacher recruitment; to teacher pride, identification and ownership in the
school. Using a 4-point Likert-type scale labeled “Agree”, 4, “Tend to Agree”, 3, “Tend to
Disagree”, 2, and “Disagree”, 1, respondents were asked to rate their attitude toward
differentiated compensation. One form of the survey was administered to the teachers and
mentor teacher-achievement coaches, and another parallel survey was administered to the
principals and superintendents. To ascertain the overall measurement quality of the instrument,
based on group characteristics, principals and superintendents were pooled to form an
administrator subgroup. An initial assessment of the responses indicated the pooled
administrator group (n = 73) was indeed homogeneous. A pooling of the two teacher subgroups
An Argument for Rasch Analysis 9
was not performed, because the mentor teacher-achievement coaches were determined to be
positioned between teachers and administrators. Thus, Rasch analysis was conducted for three
groups: pooled administrators, teachers, and mentor teacher-achievement coaches.
Procedure
A one-parameter Item Response Theory model was utilized, commonly known as the
Rasch Model, using Winsteps software (Linacre, 2004 version 3.51). Winsteps implements the
Andrich “rating scale” model with the Joint Maximum Likelihood Estimation method, also
known as UCON, which does not assume a person distribution and is flexible with missing data
(Wright & Masters, 1982). The Rasch model used in Winsteps for this analysis is the
polytomous “Rating Scale” model with the equation: ( )log P P B D Fnij ni j n i−⎛⎝⎜
⎞⎠⎟ = − −1 j , where
is the probability that person n encountering item i is observed in category j, is the
“ability” or rater-severity measure of person n, is the difficulty-to-endorse measure of item i,
and is the “calibration” measure of category j relative to category
Pnj Bn
Di
Fj ( )j −1 (Linacre, 2004).
The Rasch model uses the sum of the item ratings simply as a starting point for
estimating probabilities of those responding. Because it is based upon the ability to endorse a set
of items and the difficulty of a set of items, it is assumed item difficulty is the main characteristic
influencing responses (Linacre, 1999). Here, two facets are involved, the instrument’s items and
the respondents. From a Rasch perspective, a respondent’s willingness to endorse interacts with
an item’s difficulty to assign a certain score to produce an observed outcome (Linacre, 2002). In
general, people are more likely to endorse easy-to-endorse items than those that are difficult to
endorse, and people with higher willingness-to-endorse scores are more agreeable than those
with low scores.
An Argument for Rasch Analysis 10
Rasch analysis reports person willingness-to-endorse and item difficulty-to-endorse
estimates along a logit (log odds unit) scale, “a unit interval scale in which the unit intervals
between the locations on the person-item map have a consistent value or meaning” (Bond and
Fox, 2001, p. 29). Bond and Fox explain that employing Rasch techniques allows for the
ordering of respondents along this continuum of willingness to endorse items and orders items
along a continuum according to their difficulty to endorse. “Based on this logic of order, the
Rasch analysis software programs perform a logarithmic transformation on the item and person
data to convert the ordinal data to yield interval data…actual item and person performance
probabilities determine the interval sizes” (p. 29).
Review of the instrument began with a examination of various tables displaying the raw
scores, measures, and ZSTD scores for the survey items and the respondents (See Tables 1, 5,
and 9). ZSTD scores are mean-square fit statistics standardized to approximate a theoretical
mean 0 and standard deviation 1. INFIT ZSTDs are sensitive to irregular inlying patterns and
OUTFIT ZSTDs are sensitive to unexpected rare extremes. Survey items and respondents that
did not adequately fit the model requirements were identified using the ZSTD scores, with a
cutoff set at 2. While there is not a specific rule defining the cutoff, the commonly accepted
interpretation is that infit and outfit values greater than +2 or less than –2 indicate less
compatibility with the model than expected (Bond and Fox, 2001). The fit statistics for specific
items (see Tables 4, 8, and 12) are used to mark items and persons for further scrutiny, with the
recommendation that certain items be rewritten or excluded for future analyses, instead of
blindly interpreting the total raw score for all persons on all items as the total construct measure.
±
Diagnostic statistics produced by Winsteps were reviewed as a way to understand, assess
and suggest improvements to the measurement, or data collection, system (Linacre, 2004). The
An Argument for Rasch Analysis 11
output includes (1) item polarity (see Tables 2, 6 and 10), which under the model should reveal
all positive correlations; (2) empirical item-category measures (see Figure 1, 5 and 9), displaying
if category values correspond to ‘more’ of a variable to the right; (3) category function (see
Tables 3, 7 and 11), which allows the researcher to determine if rating scale categories advance;
and (4) a graph of probability modes (see Figures 2, 6 and 10), which indicates if each category
is the most probable category at some point.
A person and item map is available for review in Figures 3, 7 and 11. These maps set
respondents and survey items against each other so the developer can clearly identify which
items are more difficult to endorse and which respondents are more agreeable. Gaps suggest
items could be added to better span the variable. Furthermore, item difficulty-to-endorse is
matched to the overall agreeability of the sample. Thus, if the map revealed the mass of the
survey items above the mass of the respondents, it would indicate the questions were overall
difficult to endorse for the corresponding sample. This is a signal to the instrument developer
that additional items might need to be constructed to be positioned lower on the hierarchy or that
would be easier for the respondents to endorse. Making these changes would allow for a better
picture of the attitudes of the sample on the construct.
The initial Rasch analyses were run using the data as they were coded upon initial
collection. A review of the item polarity identified certain items having negative correlations
across most of the groups. The items were 3: A differentiated compensation program will have a
negative impact on the morale of teachers in the system; 12: Relations between administrative
and instructional staff will be negatively affected if a differentiated compensation program is
adopted; 2: Differentiated compensation would not enhance the positive relationship among
teachers and administrators; 8: Teachers receiving differentiated compensation will be less
An Argument for Rasch Analysis 12
cooperative with their peers; and 7: Linking teacher salary to student achievement on
standardized tests has no place in education. Since the wording of these items encouraged a
reverse use of the scale by respondents, these items were reverse-coded in the Rasch analyses to
compensate for the negative wording and improve the diagnostics. Item 17: Non-certified
teachers should pay all the costs of becoming certified, was negatively correlated for some
groups and had high outfit ZSTD scores prior to recoding, so it was recoded as well. Still, it
could be argued that this item is better left with the original coding. Item 33: There is too much
peer pressure here to do a good job, was not reverse coded but could be viewed as a negative
statement by some respondents. Thus, items 17 and 33 are two items that should be reviewed by
the survey construction team.
RESULTS
The survey data collected from the pooled group of administrators indicates a stable
measurement instrument. The summary statistics (see Table 1) reveal an acceptable person
reliability of .87 and item reliability of .95 on a scale from 0 to 1. The reliability measures can
be interpreted similarly to that of Cronbach’s alpha. Thus, with these reliability estimates, it is
reasonable to assume that another group of respondents with a similar average agreeability
measure would produce similar results.
All items have positive correlations as displayed in the Item Polarity output (see Table 2);
however, the empirical item-category measures output (Figure 1) reveals certain items’ category
values do not function with category values corresponding to “more” of the next variable. These
items are 33: There is too much peer pressure here to do a good job; 19: Certified teachers are
more effective in the classroom than non-certified teachers; 32: Teachers believe their school
stresses excellence; 29: Teachers take pride in being a part of their school; and 16: All teachers
An Argument for Rasch Analysis 13
should be required to be certified to teach. As displayed in Table 3, average measures for the
rating scale categories do advance, and no category is especially noisy. Still, categories 3 and 4
on the scale comprise 87% of the total responses, illustrating the 4-point scale is functioning as a
nearly dichotomous scale. The category probabilities graph (see Figure 2) reveals that category
2 is almost flat, when it is expected that each category should have a distinct peak. This finding
reveals that the “Tend to Disagree” choice does not aid in defining a distinct point on the
variable (Bond and Fox, 2001). Finally, the Person-Item map (Figure 3) reveals the mean
measure for the survey items is well below the mean measure for the respondents, indicating the
items are generally easy to endorse for the administrators, which are agreeable to the constructed
items.
For the group of pooled administrators, no category has extremely large INFIT or
OUTFIT ZSTD scores (see Table 4). The largest value reported is for item 17: Non-certified
teachers should pay all the costs of becoming certified, which has an OUTFIT ZSTD of 4.0.
This is followed by item 19: Certified teachers are more effective than non-certified teachers,
with an OUTFIT ZSTD of 3.0. Five items have a negative OUTFIT ZSTD greater than the
traditional cutoff of –2, but none is less than –3.5. Negative OUTFIT ZSTD suggests less
variation than the probabilistic model would predict. Given this is a group of administrators
within the same state, sharing common issues, it is not surprising their responses might be
predictable.
The teacher survey Rasch analysis reveals a less effective measurement system or survey
instrument. Summary statistics presented in Table 5 indicate teachers are generally less
agreeable on the items than the administrators. The mean teacher measure is 1.24 logits with a
standard deviation of .75, while the mean administrator measure is 1.74 with a standard deviation
An Argument for Rasch Analysis 14
of .95. Person reliability is .84 and item reliability is high at .99, so the instrument is functioning
acceptably overall, although there are concerns to be addressed.
Looking to the Item Polarity Table (see Table 6), Item 33: There is too much peer
pressure to do a good job, stands apart as having a negative correlation, indicating this item does
not point in the same direction as the other items. Item 33 also appears on the empirical item-
category measures chart (Figure 5) as having category values ordered 2-3-4-1. Other items with
out of line category values are item 19: Certified teachers are better qualified to teach than non-
certified teachers; item 16: All teachers should be certified; and item 31: I feel a sense of
ownership in this school. Overall, the category average measures advance, as displayed in Table
7, although respondents select 3 and 4 most often. Here again, category 2 is relatively flat, as
shown in Figure 6. It is possible to collapse categories 1 and 2; this would result in probability
curves that show each category represents a distinct portion of the variable and would likely
improve the rating scale diagnostics. However, unless the survey is reconstructed to demonstrate
such collapsing, it is not recommended, as interpretation of results might be altered, causing a
misinterpretation of the original responses. As with the administrators, the Person-Item map
(Figure 7) reveals the mean measure for the survey items is well below the mean measure for the
respondents, indicating the items are generally easy to endorse for this group, which is agreeable
to the constructed items.
There are multiple misfitting items for the teachers. Eleven items have an OUTFIT
ZSTD greater than 2 and as high as 9.9, and 14 items have OUTFIT ZSTD less than –2. Items
with high OUTFIT ZSTD scores are highlighted in Table 8. OUTFIT ZSTD scores greater than
2 indicate unexpected and unrelated irregularities in the responses. There are many plausible
causes for this, including ambiguous wording, negative wording and debatable or misleading
An Argument for Rasch Analysis 15
options (Linacre, 2004). Twelve items have OUTFIT ZSTD less than –2. Items with OUTFIT
less than –2 include 20: Differentiated compensation programs help recruit teachers who can
improve student learning and 11: Students’ standardized test scores in the district would improve
if a differentiated compensation program were adopted. As previously noted, large negative fit
statistics indicate there is less variability than the model would predict. This can be caused by
redundancy among items or could be a reflection of high agreement across respondents.
The instrument, again, works well with the mentor teacher-achievement coach
respondents, but discrepancies exist. Summary statistics displayed in Table 9 indicate the
mentor teachers have a mean measure of 1.40 with a standard deviation of .71, placing them
between the teachers and the administrators, as previously hypothesized. Person reliability for
this group is .83 and item reliability is .96; again reasonable measures.
The Item-Polarity Table (see Table 10) indicates item 19: Certified teachers are more
effective in the classroom than non-certified teachers, points in the opposite direction of the other
items. As displayed in Figure 9, items with categories out of the expected 1-2-3-4 order are 33:
There is too much peer pressure here to do a good job (4-1-2-3); 11: Students’ standardized test
scores in the district would improve if differentiated a compensation program were adopted (1-2-
4-3); 20: Differentiated compensation programs help recruit teachers who can improve student
learning (2-1-3-4); 25: The size of the salary bonus I could receive to become certified in a
critical shortage area or “difficult assignments” must be large enough to motivate me (2-1-3-4);
13: School districts should pay for university coursework in content areas (1-2-4-3); 19: Certified
teachers are more effective in the classroom than non-certified teachers (3-4-2); 32: This school
stresses excellence (3-4-2); 30: I feel a sense of ownership in this school (3-4-2); and 31: I feel a
sense of ownership in student learning (3-4-2). As displayed in Table 11, average measures for
An Argument for Rasch Analysis 16
the rating scale categories do advance, and no category is especially noisy. Still, here again
categories 3 and 4 on the scale dominate the responses, comprising 80% of the total responses,
illustrating that the 4-point scale is again functioning as a nearly dichotomous scale. Also, the
category probabilities graph (see Figure 10) reveals that category 2 is almost flat, when it is
expected that each category should have a distinct peak.
Fit statistics for the mentor teacher-achievement coach group are much better than for the
teachers or administrators (see Table 12). Item 33: There is too much peer pressure here to do a
good job, is again flagged as misfitting but with an OUTFIT ZSTD of only 3.1. Only two other
survey items have an OUTFIT ZSTD of greater than 2 for this group, item 13: School districts
should pay for university coursework in content areas and item 19: Certified teachers are more
effective in the classroom than non-certified teachers. There are fewer items with high negative
OUTFIT scores as well, and the largest OUTFIT is –3.6 for item 5: A differentiated
compensation program will positively affect teacher morale. Other items with OUTFIT of less
than –2 are 3: A differentiated compensation program will have a negative impact on the morale
of teachers in the system; 10: Differentiated compensation programs recognize teacher
contributions to student learning; 20: Differentiated compensation programs help recruit teachers
who can improve student learning; and 12: Relations between administrative and instructional
staff will be negatively affected if a differentiated compensation program is adopted.
CONCLUSIONS
In analyzing results collected via a survey instrument, it is presumed the respondents
have an accurate perception of the construct, rate items according to reproducible criteria, and
accurately record their ratings within uniformly spaced levels. In fact, as noted in Wright (1997),
ratings are simply responses based on fluctuating personal criteria, the responses are not always
An Argument for Rasch Analysis 17
interpreted as intended or recorded correctly, and these ratings are ordinal so they do not add up
to measures. Rasch analysis produces measures, provides a basis for insight into the quality of
the measurement tool and provides information to allow for systematic diagnosis of misfit.
Based on the results of the study, the Differentiated Compensation Survey should be reviewed
prior to future data collection. For each of the three groups, the 4-point scale does not function
as expected; as most respondents favor categories 3 and 4. Collapsing categories 1 and 2 on the
scale would result in probability curves that indicate each category represents a distinct portion
of the variable and would likely improve the rating scale diagnostics. For future surveys, the
team might consider a restructuring of the categories, creating a more tiered construction, of
‘Disagree’, ‘Neutral’ and ‘Agree’. Additionally, a few items should be given further attention
before being used in subsequent evaluations. Item 33 is highlighted throughout each of the
analyses, and it is recommended this item be set aside for the statistical analysis and to consider
rewording it prior to future data collection.
Finally, it is important for the users of the results of the survey to review the hierarchy of
items as displayed in Figures 4, 8, and 12 to determine if they are meaningful. The items most
difficult to endorse (or to agree with) are found at the top end of the figure; the items easiest to
agree with are found at the bottom. Inconsistencies between the hierarchies as perceived by the
respondents and as expected based on the literature must be examined, with consideration that
the source of the problem might stem from either the theoretical basis or the empirical results
(Bond and Fox, 2001.) EDUCATIONAL IMPORTANCE
Within the field of education, the development of instruments to assess affective domain
constructs has been a problematic area (Aiken, 1996; Martin, 1983). Surveys are the most
An Argument for Rasch Analysis 18
common example of self-reported data collection and continue to be one of the most popular
research methodologies for graduate studies and published papers in education (Aiken, 1988;
Babbie, 1992; Gay, 1981). Even so, the efficiency and effectiveness of the instrument as a
measurement tool is often overlooked or underemphasized.
As noted by Sampson and Bradley (2003) the classical test theory model produces a
descriptive summary based on statistical analysis, but it is limited if not absent of the capability
to assess the quality of the instrument. It is important to begin at the level of measurement and to
identify weaknesses that may limit the reliability and validity of the measures made with the
instrument. As indicated in the study, Rasch analysis tackles many of the deficiencies of the
classical test theory model in that it has the capacity to incorporate missing data, produces
validity and reliability measures for person measures and item calibrations, measures persons
and items on the same metric, and is person and sample-free.
The survey development team, along with researchers, organizations or institutions
analyzing similar rating scale data will benefit from the results of this study as it provides a
sound methodology for analyzing such data. The education community will also benefit by
receiving better-informed results by collecting data using a more valid and reliable instrument.
An Argument for Rasch Analysis 19
An Argument for Rasch Analysis 20
References
Aiken, L. R. (1988). The problem of non-response in survey research. The Journal of
Experimental Education, 56, 116-119.
Aiken, L. (1996). Rating scales and checklists: Evaluating behavior, personality, and
attitudes. John Wiley & Sons, Inc.
Babbie, E. (1992). The practice of social research (6th ed). Belmont, CA: Wadsworth.
Bond, T., & Fox, C. (2001). Applying the Rasch model: Fundamental measurement in the
human sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
Gay, L. R. (1981). Educational research: competencies for analysis & application.
Columbus, OH: Charles E. Merrill.
Linacre, J. (1999). A User’s Guide to Facets Rasch Measurement Computer Program.
Chicago, IL: MESA Press.
Linacre, J. (2002). Facets, factors, elements and levels [Electronic version]. Rasch
Measurement Transactions, 16 (2), 880.
Linacre, J. (2004). A User’s Guide to Winsteps Rasch-Model Computer Programs.
Chicago, IL: MESA Press.
Linacre, J. (2004). Winsteps (Version 3.51) [Computer software]. Retrieved from
www.winsteps.com.
Martin, R. (1983). Temperament: A review of research with implications for the school
psychologist. School Psychology Review, 12(3), 266-273.
Michell, J. (1997). Quantitative science and the definition of measurement in
psychology. British Journal of Psychology, 88(3), 355-383.
Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests.
An Argument for Rasch Analysis 21
Chicago: The University of Chicago Press (original work published in 1960).
Sampson, S. & Bradley, K. D. (November, 2003). Rasch analysis of educator supply and
demand rating scale data [Electronic Version]. Research Methods Forum.
Available at: http://aom.pace.edu/rmd/2003forum.html
Smith, E., Jr. (2000). Rasch Measurement Models. Paper presented at An Introduction to
Rasch Measurement: Theory and Applications, Chicago.
Wright, B. (1997). Fundamental measurement for outcome evaluation [Electronic
version]. Physical Medicine And Rehabilitation: State Of The Art Reviews, 11(2),
261-288. Available at: http://www.rasch.org/memo66.htm
Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago, IL: MESA
Press.
Wright, B. D. & Stone, M. H. (1979). Best test design. Chicago: MESA Press.
An Argument for Rasch Analysis 22
APPENDIX A: MENTOR TEACHER-ACHIEVEMENT COACH SURVEY
KENTUCKY DIFFERENTIATED TEACHER COMPENSATION STUDY
MENTOR TEACHER-ACHIEVEMNT COACH SURVEY
The Kentucky Department of Education (KDE) recently funded 10 school districts to plan and pilot a differentiated teacher compensation program to determine policy implications for: (1) Recruiting and retaining teachers in critical shortage areas; (2) Reducing the number of emergency certified teachers; (3) Providing incentives for teachers to serve in difficult assignments and hard-to-fill positions; (4) Providing voluntary career advancement opportunities; and (5) Rewarding teachers who increase their knowledge and skills. An important part of this work is to ascertain the effectiveness of these different approaches over a two-year period (2003-2005). KDE has asked the College of Education at the University of Kentucky to evaluate the Differentiated Compensation Pilot Site program.
You are being invited to complete the attached survey because your opinions and ideas about the differentiated teacher compensation are valuable. Your participation in completing the enclosed survey is voluntary and confidential. If you chose to not participate, you are not subject to any penalty. By returning a completed survey you are giving your consent to researchers at the University of Kentucky to use this data. In reporting results, no individual, school or district identity will be divulged. Only group statistical responses will be aggregated for reports or publications. The task should take no more than 15 minutes. Your insight and assistance will be greatly appreciated. This survey has been reviewed and approved by the University of Kentucky Office of Research Integrity. Directions: 1. Please provide a response to every question. If none of the alternatives provided
for a question corresponds exactly to your position or opinion, select the alternative that comes closest to the answer you would like to give.
2. To complete the survey, follow the directions for each section. If you change a
response, be sure that the change is legible and clear. 3. Place your completed questionnaire in the envelope provided and mail it by
March 15, 2004.
Thank you for assisting the Kentucky State Department of Education with this survey!
An Argument for Rasch Analysis 23
KENTUCKY DIFFERENTIATED TEACHER COMPENSATION STUDY
MENTOR TEACHER - ACHIEVEMENT COACH SURVEY
Definition: “Differentiated compensation programs” include a range of incentives that are added on to present compensation including: salary bonuses for teaching in critical shortage areas (math, science, special ed., or foreign language) or serving in “difficult assignments” or “hard-to-fill positions” in rural and urban districts; financial support for seeking advanced degrees or certification; participating in voluntary career advancement opportunities or receiving rewards for teachers who increase their knowledge and skills to improve teaching and learning.
PART 1
District?:_______________________ School?__________________________
1. What is your age (years)? _____ 2. What is your gender? [ ] Male [ ] Female 3. What is your racial/ethnic group? [ ] African American [ ] Asian American [ ] Caucasian [ ] Pacific Islander [ ] Hispanic American [ ] Other (Specify): [ ] Native American _____________ 4. Your marital status is? [ ] Married [ ] Single 5. What is the highest degree that you have earned? [ ] Bachelor’s Degree [ ] Specialist [ ] Master’s Degree [ ] Doctor’s Degree 6. How many years ago did you receive your highest academic degree? ___________ 7. What is your highest salary rank? [ ] Rank III [ ] Rank II [ ] Rank I 8. Your present contract salary is? $_______ 9. How many years ago did you attain your highest salary rank? _____________ 10. How many years have you taught? _____ 11. How many years have you been teaching in your present school? ______ 12. At what level are you presently working? [ ] Elementary school [ ] Tech. Center [ ] Junior high [ ] District Office [ ] Middle school [ ] Other___________ [ ] High school
13. What is your primary teaching area? [ ] History/Soc Studies [ ] Special education [ ] English [ ] Math [ ] Reading [ ] Science [ ] Career Tech [ ] Other ________ [ ] Foreign Language 14. Are you receiving bonus pay to serve as a mentor teacher or achievement coach? [ ] Yes [ ] No 15. How much is the salary bonus you are receiving for to serve as a mentor teacher or achievement coach? $ __________ [ ] Not Applicable 16. Will a salary bonus motivate you to continue to serving as a mentor teacher or achievement coach? [ ] Yes [ ] No 17. Check the year(s) you have served as a mentor teacher or achievement coach. [ ] 2003-2004 [ ] 2004-2005 18. In what area are you serving as a mentor or achievement coach? [ ] Special education [ ] Foreign Language [ ] Math [ ] Science [ ] Administration 19. How many students were enrolled in your school as of September 2003? [ ] fewer than 299 [ ] 1,500 to 1,799 [ ] 300 to 599 [ ] 1,800 to 2,099 [ ] 600 to 899 [ ] 2,100 to 2,399 [ ] 900 to 1,199 [ ] 2,400 to 2,699
An Argument for Rasch Analysis 24
20. Your district is (check one): [ ] Rural [ ] Suburban [ ] Small Town [ ] Urban
An Argument for Rasch Analysis 25
Part 2
Directions: How much do you agree or disagree with each of the following statements? (Please circle your response to each item) 1. A differentiated compensation program will help recruit better-qualified people to the teaching profession.
Agree Tend to agree Tend to disagree
Disagree
2. Differentiated compensation would not enhance the positive relationship among teachers and administrators.
Agree Tend to agree Tend to disagree Disagree
3. A differentiated compensation program will have a negative impact on the morale of teachers in the system.
Agree Tend to agree Tend to disagree Disagree
4. A differentiated compensation program helps retain teachers in critical shortage areas.
Agree Tend to agree Tend to disagree Disagree
5. A differentiated compensation program will positively affect teacher morale.
Agree Tend to agree Tend to disagree Disagree
6. A differentiated compensation program provides incentives for continuous growth aligned with professional development standards.
Agree Tend to agree Tend to disagree Disagree
7. Linking teacher salary to student achievement on standardized tests has no place in education.
Agree Tend to agree Tend to disagree Disagree
8. Teachers receiving differentiated compensation will be less cooperative with their peers.
Agree Tend to agree Tend to disagree Disagree
9. School districts should support teachers who are voluntarily advancing their careers.
Agree Tend to agree Tend to disagree Disagree
10. Differentiated compensation programs recognize teacher contributions to student learning.
Agree Tend to agree Tend to disagree Disagree
11. Students’ standardized test scores in the district would improve if a differentiated compensation program were adopted.
Agree Tend to agree Tend to disagree Disagree
12. Relations between administrative and instructional staff will be negatively affected if a differentiated compensation program is adopted.
Agree Tend to agree Tend to disagree Disagree
13. School districts should pay for university coursework in content areas.
Agree Tend to agree Tend to disagree Disagree
14. Career advancement opportunities should be aligned with school improvement goals and professional development.
Agree Tend to agree Tend to disagree Disagree
15. A differentiated compensation program will result in a higher teacher retention rate.
Agree Tend to agree Tend to disagree Disagree
16. All teachers should be required to be certified to teach.
Agree Tend to agree Tend to disagree Disagree
17. Non-certified teachers should pay all the costs of becoming certified.
Agree Tend to agree Tend to disagree Disagree
18. When non-certified teachers are hired, school districts should provide support for them to become certified.
Agree Tend to agree Tend to disagree Disagree
An Argument for Rasch Analysis 26
19. Certified teachers are more effective in the classroom than non-certified teachers.
Agree Tend to agree Tend to disagree Disagree
20. Differentiated compensation programs help recruit teachers who can improve student learning.
Agree Tend to agree Tend to disagree Disagree
21. It is appropriate for teachers to receive salary bonuses to teach in critical shortage areas including math, science, special ed., or foreign language.
Agree Tend to agree Tend to disagree Disagree
22. It is appropriate for teachers to receive salary bonuses to serve in urban schools classified as a “difficult assignment” or “hard-to-fill” positions.
Agree Tend to agree Tend to disagree Disagree
23. It is appropriate for teachers to receive salary bonuses to serve in rural schools classified as a “difficult assignment” or “hard-to-fill” position.
Agree Tend to agree Tend to disagree Disagree
24. If I receive a differentiated compensation (salary) bonus I deserve it.
Agree Tend to agree Tend to disagree Disagree
25. The size of the salary bonus I could receive to become certified in a critical shortage area or “difficult assignments” must be large enough to be motivate me.
Agree Tend to agree Tend to disagree Disagree
26. A salary bonus motivates me to improve my knowledge and skills.
Agree Tend to agree Tend to disagree Disagree
27. Improving my knowledge and skills enhances student learning.
Agree Tend to agree Tend to disagree Disagree
28. I identify with this school. Agree Tend to agree Tend to disagree Disagree 29. I take pride in being a part of this school. Agree Tend to agree Tend to disagree Disagree 30. I feel a sense of ownership in this school. Agree Tend to agree Tend to disagree Disagree 31. I feel a sense of ownership in student learning.
Agree Tend to agree Tend to disagree Disagree
32. This school stresses excellence. Agree Tend to agree Tend to disagree Disagree 33. There is too much peer pressure here to do a good job.
Agree Tend to agree Tend to disagree Disagree
34. I’m encouraged to make suggestions about how we can be more effective.
Agree Tend to agree Tend to disagree Disagree
Thank you for completing this survey.
An Argument for Rasch Analysis 27
APPENDIX B: SUPERINTENDENT SURVEY
KENTUCKY DIFFERENTIATED TEACHER COMPENSATION STUDY
SUPERINTENDENT SURVEY
The Kentucky Department of Education (KDE) recently funded 10 school districts to plan and pilot a differentiated teacher compensation program to determine policy implications for: (1) Recruiting and retaining teachers in critical shortage areas; (2) Reducing the number of emergency certified teachers; (3) Providing incentives for teachers to serve in difficult assignments and hard-to-fill positions; (4) Providing voluntary career advancement opportunities; and (5) Rewarding teachers who increase their knowledge and skills. An important part of this work is to ascertain the effectiveness of these different approaches over a two-year period (2003-2005). KDE has asked the College of Education at the University of Kentucky to evaluate the Differentiated Compensation Pilot Site program.
You are being invited to complete the attached survey because your opinions and ideas about the differentiated teacher compensation are valuable. Your participation in completing the enclosed survey is voluntary and confidential. If you chose to not participate, you are not subject to any penalty. By returning a completed survey you are giving your consent to researchers at the University of Kentucky to use this data. In reporting results, no individual, school or district identity will be divulged. Only group statistical responses will be aggregated for reports or publications. The task should take no more than 15 minutes. Your insight and assistance will be greatly appreciated. This survey has been reviewed and approved by the University of Kentucky Office of Research Integrity. Directions: 1. Please provide a response to every question. If none of the alternatives provided
for a question corresponds exactly to your position or opinion, select the alternative that comes closest to the answer you would like to give.
2. To complete the survey, follow the directions for each section. If you change a
response, be sure that the change is legible and clear. 3. Place your completed questionnaire in the envelope provided and mail it by
March 15, 2004. Thank you for assisting the Kentucky State Department of Education with this survey!
An Argument for Rasch Analysis 28
KENTUCKY DIFFERENTIATED TEACHER COMPENSATION STUDY
SUPERINTENDENT SURVEY
Definition: “Differentiated compensation programs” include a range of incentives that are added on to present compensation including: salary bonuses for teaching in critical shortage areas (math, science, special ed., or foreign language) or serving in “difficult assignments” or “hard-to-fill positions” in rural and urban districts; financial support for seeking advanced degrees or certification; participating in voluntary career advancement opportunities or receiving rewards for teachers who increase their knowledge and skills to improve teaching and learning.
PART 1 District:________________________ 1. What is your age (years)? _____ 2. What is your gender? [ ] Male [ ] Female 3. What is your racial/ethnic group? [ ] African American [ ] Asian American [ ] Caucasian [ ] Pacific Islander [ ] Hispanic American [ ] Other (Specify): [ ] Native American _____________ 4. Your marital status is? [ ] Married [ ] Single 5. What is the highest degree that you have earned? [ ] Bachelor’s Degree [ ] Specialist [ ] Master’s Degree [ ] Doctor’s Degree 6. How many years ago did you receive your highest academic degree? ___________ 7. What is your highest salary rank? [ ] Rank III [ ] Rank II [ ] Rank I 8. Your present contract salary is? $_______ 9. How many years ago did you attain your highest salary rank? _____________ 10. How many years have you been a superintendent? _____ 11. How many years have you been a superintendent in your present district? ______ 12. In how many districts have you served as superintendent? _____ 14. How many students were enrolled in your district as of September 2003?
[ ] 1,000 to 1,999 [ ] 6,000 to 7,499 [ ] 2,000 to 3,499 [ ] 7,500 to 9,999 [ ] 3,500 to 4,499 [ ] 10,000 to 11,999 [ ] 4,500 to 4,999 [ ] 12,000 to 14,999 [ ] 5,000 to 5,999 [ ] 95,000 t0 100,000 15. Your district is (check one): [ ] Rural [ ] Suburban [ ] Small Town [ ] Urban 16. Check the year(s) you have been involved with the differentiated compensation program in your district. [ ] 2003-2004 [ ] 2004-2005
Rating Scale Quality 29
Part 2 Directions: How much do you agree or disagree with each of the following statements? (Please circle your response to each item) 1. A differentiated compensation program will attract better-qualified people to the teaching profession.
Agree Tend to agree Tend to disagree
Disagree
2. Differentiated compensation would not enhance the positive relationship among teachers and administrators.
Agree Tend to agree Tend to disagree Disagree
3. A differentiated compensation program will have a negative impact on the morale of teachers in the system.
Agree Tend to agree Tend to disagree Disagree
4. A differentiated compensation program helps retain teachers in critical shortage areas.
Agree Tend to agree Tend to disagree Disagree
5. A differentiated compensation program will positively affect teacher morale.
Agree Tend to agree Tend to disagree Disagree
6. A differentiated compensation program provides incentives for continuous growth aligned with professional development standards.
Agree Tend to agree Tend to disagree Disagree
7. Linking teacher salary to student achievement on standardized tests has no place in education.
Agree Tend to agree Tend to disagree Disagree
8. Teachers receiving differentiated compensation will be less cooperative with their peers.
Agree Tend to agree Tend to disagree Disagree
9. School districts should support teachers who are voluntarily advancing their careers.
Agree Tend to agree Tend to disagree Disagree
10. Differentiated compensation programs recognize teacher contributions to student learning.
Agree Tend to agree Tend to disagree Disagree
11. Students’ standardized test scores in the district would improve if a differentiated compensation program were adopted.
Agree Tend to agree Tend to disagree Disagree
12. Relations between administrative and instructional staff will be negatively affected if a differentiated compensation program is adopted.
Agree Tend to agree Tend to disagree Disagree
13. School districts should pay for university coursework in content areas.
Agree Tend to agree Tend to disagree Disagree
14. Career advancement opportunities should be aligned with school improvement goals and professional development.
Agree Tend to agree Tend to disagree Disagree
15. A differentiated compensation program will result in a higher teacher retention rate.
Agree Tend to agree Tend to disagree Disagree
16. All teachers should be required to be certified to teach.
Agree Tend to agree Tend to disagree Disagree
17. Non-certified teachers should pay all the costs of becoming certified.
Agree Tend to agree Tend to disagree Disagree
18. When non-certified teachers are hired, school districts should provide support for them to become certified.
Agree Tend to agree Tend to disagree Disagree
Rating Scale Quality 30
19. Certified teachers are more effective in the classroom than non-certified teachers.
Agree Tend to agree Tend to disagree Disagree
20. Differentiated compensation programs help recruit teachers who can improve student learning.
Agree Tend to agree Tend to disagree Disagree
21. It is appropriate for teachers to receive salary bonuses to teach in critical shortage areas including math, science, special ed., or foreign language.
Agree Tend to agree Tend to disagree Disagree
22. It is appropriate for teachers to receive salary bonuses to serve in urban schools classified as a “difficult assignment” or “hard-to-fill” positions.
Agree Tend to agree Tend to disagree Disagree
23. It is appropriate for teachers to receive salary bonuses to serve in rural schools classified as a “difficult assignment” or “hard-to-fill” position.
Agree Tend to agree Tend to disagree Disagree
24. If teachers receive a differentiated compensation (salary) bonus they deserve it.
Agree Tend to agree Tend to disagree Disagree
25. The size of the salary bonus teachers receive to become certified in a critical shortage area or “difficult assignments” must be large enough to motivate them.
Agree Tend to agree Tend to disagree Disagree
26. A salary bonus motivates teachers to improve their knowledge and skills.
Agree Tend to agree Tend to disagree Disagree
27. Improving teachers’ knowledge and skills enhances student learning.
Agree Tend to agree Tend to disagree Disagree
28. Teachers identify with their school. Agree Tend to agree Tend to disagree Disagree 29. Teachers take pride in being a part of their school.
Agree Tend to agree Tend to disagree Disagree
30. Teachers feel a sense of ownership in their school.
Agree Tend to agree Tend to disagree Disagree
31. Teachers feel a sense of ownership in student learning.
Agree Tend to agree Tend to disagree Disagree
32. Teachers believe that their school stresses excellence.
Agree Tend to agree Tend to disagree Disagree
33. There is peer pressure among teachers to do a good job.
Agree Tend to agree Tend to disagree Disagree
34. Teachers are encouraged to make suggestions about how they can be more effective.
Agree Tend to agree Tend to disagree Disagree
Thank you for completing this survey. Note: Other surveys, as discussed in the body of the paper are redundant with those
included in the appendices.
Rating Scale Quality 31
APPENDIX C: OUTPUT FOR POOLED ADMINISTRATORS Table 1 Summary Statistics for administrators Measure Infit ZSTD Outfit ZSTD Person M 1.74 .0 -.1 S.D. .95 1.6 1.6 Item M .00 -.1 -.1 S.D. 1.07 1.8 1.8 Real person reliability .87 Real item reliability .95
Table 2 Item polarity for administrators Entry Number Pt. Measure
Correlation Item
16 .00 All teachers should be required… 19 .00 Certified teachers are more effective… 34 .19 Teachers are encouraged to make suggestions 17 .20 Non-certified teachers are hired… 33 .22 There is too much peer pressure… 18 .29 When non-certified teachers are hired… 32 .32 Teachers believe their school stresses excellence… 7 .37 Linking teacher salary to… 2 .37 Differentiated compensation would not enhance… 27 .38 Improving knowledge and skills… 31 .42 Teachers feel a sense of ownership in student… 13 .42 School districts should pay… 9 .42 School districts should… 29 .47 Teachers take pride in being a part… 28 .49 Teachers identify with their school… 3 .52 Differentiated compensation will have a negative… 14 .52 Career advancement opportunities… 30 .53 Teachers feel a sense of ownership in school… 25 .57 The size of the salary bonus… 10 .57 Differentiated compensation programs recognize teacher… 1 .60 Differentiated compensation will attract better quality… 4 .62 Differentiated compensation helps retain teachers in critical… 12 .64 Relations between administrators… 15 .64 A differentiated compensation program will result in higher retention… 8 .65 Differentiated compensation will positively affect… 5 .65 Differentiated compensation will positively affect… 26 .65 A salary bonus motivates teachers… 11 .65 Students’ standardized test scores… 21 .66 It is appropriate for... bonuses in critical shortage… 22 .67 It is appropriate for… bonuses in urban… 23 .67 It is appropriate for… bonuses in rural… 6 .71 Differentiated compensation provides incentives for… 20 .73 Differentiated compensation programs help recruit teachers… 24 .73 If teachers receive a differentiated compensation bonus… deserve it
Rating Scale Quality 32
Figure 1 Empirical item-category measures for administrators OBSERVED AVERAGE MEASURES FOR PERSONS (BY OBSERVED CATEGORY) -2 -1 0 1 2 3 4 5 |-------+-------+-------+-------+-------+-------+-------| NUM ITEM | 12 3 4 | 17 Non-cert Ts should pay all... | 1 23 4 | 7 linking teacher salary to... | 1 2 34 | 13 School districts should pa... | 1 2 3 4 | 11 Students' standardized tes... | 1 2 3 4 | 2 DC would not enhance the p... | 1 2 34 | 18 When non-cert Ts are hired... | 1 2 3 4 | 24 If Ts receive a DC bonus...deserve it | 1 2 3 4 | 21 It is app. for...bonuses in critical... | 1 2 3 4 | 3 DC will have a negative im... | 1 2 3 4 | 5 DC will positively affect... | 1 2 3 4 | 22 It is app. for...bonuses in urban... | 1 2 3 4 | 15 A DC program will result in higher retention... | 1 2 3 4 | 23 It is app. for...bonuses in rural... | 1 2 3 4 | 12 Relations between administ... | 1 2 3 4 | 26 A salary bonus motivates T... | 1 2 3 4 | 20 DC programs help recruit T... | 1 2 3 4 | 4 DC helps retain Ts in crit... | 3 24 | 33 There is too much peer pre... | 1 2 3 4 | 25 The size of the salary bon... | 342 | 19 cert Ts are more effective... | 1 2 3 4 | 8 Ts receiving differentiate... | 1 2 3 4 | 1 DC will attract better qua... | 1 2 3 4 | 10 DC programs recognize teac... | 1 2 3 4 | 14 Career advancement opportu... | 1 2 3 4 | 6 DC provides incentives for... | 3 2 4 | 32 Ts believe their school stresses excellence... | 3 4 | 31 Ts feel a sense of ownership in student lrng | 3 4 | 28 Ts identify with their sch... | 3 2 4 | 29 Ts take pride in being a p... | 3 4 | 30 Ts feel a sense of ownership in school | 3 4 | 27 Improving knowledge and sk... | 1 3 4 | 9 school districts should su... | 3 4 | 34 Ts are encouraged to make suggestions | 143 | 16 All Ts should be required... |-------+-------+-------+-------+-------+-------+-------| NUM ITEM -2 -1 0 1 2 3 4 5 1 2 2423227367 34442512 212 1 PERSONS T S M S T
Table 3 Category function for administrators Category Label Category Measure 1 -2.54 2 -.96 3 .70 4 2.95
Rating Scale Quality 33
Figure 2 Probability modes for administrators ++---------+---------+---------+---------+---------+---------++ R 1.0 + + O | | B | | A |111 | B .8 + 11 + I | 111 444| L | 11 44 | I | 11 44 | T .6 + 1 33333333333 44 + Y | 11 33 333 44 | .5 + 1 33 33*4 + O | 11 33 44 33 | F .4 + 1222 33 4 33 + | 2222211 **2222 44 333 | R | 222 * 222 44 33 | E | 222 33 11 22 444 333| S .2 + 2222 33 11 22244 + P |2222 333 11 4442222 | O | 333 1***4 2222 | N | 3333333 444444 111111 22222222 | S .0 +****4444444444444444444 11111111111111*********+ E ++---------+---------+---------+---------+---------+---------++ -3 -2 -1 0 1 2 3 PERSON [MINUS] ITEM MEASURE
Rating Scale Quality 34
Figure 3 Variable map for administrators MEASURE | MEASURE <more> --------------------- PERSONS -+- ITEMS --------------------- <rare> 5 + 5 | | | X | | 4 + 4 | T| XX | X | XX | 3 + 3 XXX | XXXXX S| XXXXX | X | XX XXXXXXXX |T 2 XXX + 2 | XXXXXX M| X XXXXXXXXX | XXXXXXX | X X | XX 1 XXXXXXX +S 1 XX S| XXXX | XX | XXXXX | XXXXX XX | X 0 +M XX 0 T| XXX | X | X | X | -1 +S XX -1 | XXX | | X X | XX | -2 + -2 |T X | | | | -3 + -3 | | | | | -4 + -4 <less> --------------------- PERSONS -+- ITEMS ------------------<frequent>
Rating Scale Quality 35
Table 4 M isfitting items for administrators
Entry Number
Infit ZSTD
Outfit ZSTD
Item
16 2.9 2.5 All teachers should be required… 17 4.1 4.0 Non-certified teachers should pay all… 19 2.1 3.0 Certified teachers are more effective… 2 2.6 2.8 Differentiated compensation would not enhance the… 18 2.4 2.5 When non-certified teachers are hired… 7 2.6 2.6 Linking teacher salary to… 8 -2.0 -2.1 Teachers receiving differentiated compensation… 6 -2.1 -2.5 Differentiated compensation provides incentives for… 24 -3.0 -2.9 If teachers receive a differentiated compensation bonus… 20 -2.9 -3.1 Differentiated compensation programs help recruit teachers 11 -3.8 -3.5 Students’ standardized test scores…
Figure 4 Construct key map for administrators EXPECTED SCORE: MEAN (":" INDICATES HALF-SCORE POINT) (BY OBSERVED CATEGORY) -5 -3 -1 1 3 5 7 |---------+---------+---------+---------+---------+---------| NUM ITEM 1 1 : 2 : 3 : 4 4 17 Non-cert Ts should pay all... 1 1 : 2 : 3 : 4 4 7 linking teacher salary to... 1 1 : 2 : 3 : 4 4 13 School districts should pa... 1 1 : 2 : 3 : 4 4 11 Students' standardized tes... 1 1 : 2 : 3 : 4 4 2 DC would not enhance the p... 1 1 : 2 : 3 : 4 4 18 When non-cert Ts are hired... 1 1 : 2 : 3 : 4 4 24 If Ts receive a DC bonus...deserve it 1 1 : 2 : 3 : 4 4 21 It is app. for...bonuses in critical... 1 1 : 2 : 3 : 4 4 3 DC will have a negative im... 1 1 : 2 : 3 : 4 4 5 DC will positively affect... 1 1 : 2 : 3 : 4 4 22 It is app. for...bonuses in urban... 1 1 : 2 : 3 : 4 4 15 A DC program will result in higher retention... 1 1 : 2 : 3 : 4 4 23 It is app. for...bonuses in rural... 1 1 : 2 : 3 : 4 4 12 Relations between administ... 1 1 : 2 : 3 : 4 4 26 A salary bonus motivates T... 1 1 : 2 : 3 : 4 4 20 DC programs help recruit T... 1 1 : 2 : 3 : 4 4 4 DC helps retain Ts in crit... 1 1 : 2 : 3 : 4 4 33 There is too much peer pre... 1 1 : 2 : 3 : 4 4 25 The size of the salary bon... 1 1 : 2 : 3 : 4 4 19 cert Ts are more effective... 1 1 : 2 : 3 : 4 4 8 Ts receiving differentiate... 1 1 : 2 : 3 : 4 4 1 DC will attract better qua... 1 1 : 2 : 3 : 4 4 10 DC programs recognize teac... 1 1 : 2 : 3 : 4 4 14 Career advancement opportu... 1 1 : 2 : 3 : 4 4 6 DC provides incentives for... 1 1 : 2 : 3 : 4 4 32 Ts believe their school stresses excellence... 1 1 : 2 : 3 : 4 4 31 Ts feel a sense of ownership in student lrng 1 1 : 2 : 3 : 4 4 28 Ts identify with their sch... 1 1 : 2 : 3 : 4 4 29 Ts take pride in being a p... 1 1 : 2 : 3 : 4 4 30 Ts feel a sense of ownership in school 1 1 : 2 : 3 : 4 4 27 Improving knowledge and sk... 1 1 : 2 : 3 : 4 4 9 school districts should su... 1 1 : 2 : 3 : 4 4 34 Ts are encouraged to make suggestions 11 : 2 : 3 : 4 4 16 All Ts should be required... |---------+---------+---------+---------+---------+---------| NUM ITEM -5 -3 -1 1 3 5 7 1 1 2 627733 384712212 1 PERSONS T S M S T
Rating Scale Quality 36
APPENDIX D: OUTPUT FOR TEACHERS
Table 5 Summary statistics for teachers Measure Infit ZSTD Outfit ZSTD Person M 1.24 -.1 .0 S.D. .75 1.7 1.6 Item M .00 .0 .1 S.D. 1.01 3.9 4.3 Real person reliability .84 Real item reliability .99
Rating Scale Quality 37
Table 6 Item polarity for teachers Entry Number
Pt. Measure Correlation
Item
33 -.16 There is too much peer pressure here to… 16 .07 All teachers should be required… 19 .15 Certified teachers are more effective… 17 .21 Non-certified teachers should pay all their… 13 .28 School districts should pay... 9 .28 districts should support tchrs voluntarily... 32 .29 This school stresses excellence 7 .30 linking tchr salary to...achievement... 29 .32 I take pride in being a part of this school 34 .33 I'm encouraged to make suggestions... 31 .35 I feel a sense of ownership in student learning 14 .35 Career advancement opportunities...aligned… 25 .35 The salary bonus...large enough...motivate me 28 .38 I identify with this school 30 .38 I feel a sense of ownership in this school 18 .38 When non-certified tchrs are hired, school... 27 .43 Improving my knowledge...enhances student learning 8 .45 Tchrs... will be less cooperative... 26 .50 A salary bonus motivates me to improve... 24 .53 If I receive a DC bonus I deserve it 12 .56 Relations between admin and instrctnl staff neg... 22 .60 It is appropriate...bonuses to serve in urban... 11 .60 Students’ standardized test scores would improve… 2 .61 Differentiated compensation would not enhance the positive relationship… 4 .61 Differentiated compensation helps retain teachers in critical shortage… 1 .62 Differentiated compensation helps recruit better qualified… 10 .62 Differentiated compensation programs recognize teacher contributions… 3 .62 Differentiated compensation will have a negative impact… 21 .62 It is appropriate… bonuses to teach in critical shortage areas 23 .63 It is appropriate… bonuses to serve in rural… 6 .67 Differentiated compensation provides incentives for growth 15 .67 A differentiated compensation program will result in higher retention 5 .71 Differentiated compensation will positively affect teacher morale 20 .71 Differentiated compensation programs help recruit teachers… improve…
Rating Scale Quality 38
Figure 5: Empirical item-category measures for teachers -2 -1 0 1 2 3 4 |---------+---------+---------+---------+---------+---------| NUM ITEM | 32 41 | 33 There is too much peer pressure here to... | 1 2 34 | 7 linking tchr salary to...achievement... | 1 23 4 | 17 Non-certified tchrs should pay all the costs... | 1 2 3 4 | 11 Stdnts' standardized test scores would improve... | 1 2 3 4 | 2 DC would not enhance the positive rel... | 1 2 3 4 | 18 When non-certified tchrs are hired, school... | 1 2 3 4 | 5 DC will positively affect teacher morale | 1 2 3 4 | 12 Relations between admin and instrctnl staff neg... | 1 2 3 4 | 3 DC will have a negative impact... | 1 2 3 4 | 21 It is appropriate...bonuses to teach in critical... | 1 2 3 4 | 10 DC programs recgnze teacher contributions... | 1 2 3 4 | 20 DC programs help recruit teachers...improve... | 1 2 3 4 | 15 A DC...will result in a higher retention... | 213 4 | 19 Certified teachers are more effective... | 1 2 3 4 | 23 It is appropriate...bonuses to serve in rural... | 1 23 4 | 25 The salary bonus...large enough...motivate me | 1 2 3 4 | 22 It is appropriate...bonuses to serve in urban... | 1 2 3 4 | 1 DC helps recruit better-qualified... | 1 2 3 4 | 4 DC helps retain teachers in critical shortage... | 1 2 3 4 | 26 A salary bonus motivates me to improve... | 12 3 4 | 14 Career advancement opportunities...aligned... | 1 2 3 4 | 8 Tchrs... will be less cooperative... | 1 32 4 | 13 School districts should pay... | 1 2 3 4 | 6 DC provides incentives for growth... | 21 3 4 | 34 I'm encouraged to make suggestions... | 1 2 3 4 | 24 If I receive a DC bonus I deserve it | 1 23 4 | 32 This school stresses excellence | 123 4 | 30 I feel a sense of ownership in this school | 1 2 3 4 | 28 I identify with this school | 2134 | 16 All teachers should be required to be certif... | 123 4 | 29 I take pride in being a part of this school | 1 2 3 4 | 27 Improving my knowledge...enhances student learning | 1 2 3 4 | 9 districts should support tchrs voluntarily... | 2 13 4 | 31 I feel a sense of ownership in student learning |---------+---------+---------+---------+---------+---------| NUM ITEM -2 -1 0 1 2 3 4 1 1 111221222212212111111 11 1 21 32044060898337600060750510032942 2 2 PERSONS T S M S T
Table 7 Category function for teachers Category Label Category Measure 1 -2.2 2 -.7 3 .6 4 2.37
Rating Scale Quality 39
Figure 6 Probability modes for teachers P ++---------+---------+---------+---------+---------+---------++ R 1.0 + + O | | B |11 | A | 1111 444| B .8 + 111 444 + I | 11 444 | L | 11 44 | I | 11 44 | T .6 + 1 44 + Y | 11 44 | .5 + 11 333 4 + O | 1 3333 333** | F .4 + 1 333 44 333 + | 2222**22**2 4 33 | R | 2222 *3 222 44 333 | E | 222 33 11 22*4 333 | S .2 + 2222 33 11 44 22 333 + P | 2222 33 1*4 222 3333| O |222 3333 444 111 2222 | N | 333333 444444 111111 2222222 | S .0 +********4444444444444 11111111111**********+ E ++---------+---------+---------+---------+---------+---------++ -3 -2 -1 0 1 2 3 PERSON [MINUS] ITEM MEASURE
Rating Scale Quality 40
Figure 7 Variable map for teachers MEASURE | MEASURE <more> --------------------- PERSONS -+- ITEMS --------------------- <rare> 4 + 4 | | | | | # | 3 # + 3 # | .## T| ##### | X .###### | X .######### | ###### | 2 ############ S+T 2 .##### | ################ | .############## | .############# | X .########## M| X .###################### | 1 .############# +S 1 ################# | X ############### | .############## | ######### S| XXXXX ##### | XXX #### | XXXX 0 .#### +M X 0 #### | XXXXXX ## T| X | X .# | | X . | -1 . +S -1 . | XXX | | X | X | X | X -2 +T -2 | | | | | | -3 + -3 | | | | | | -4 + -4 <less> --------------------- PERSONS -+- ITEMS ------------------<frequent> EACH '#' IN THE PERSON COLUMN IS 2 PERSONS; EACH '.' IS 1
Rating Scale Quality 41
Table 8 Item misfits for teachers Entry Number
Infit ZSTD
Outfit ZSTD
Item
33 7.2 9.6 There is too much peer pressure here to… 16 4.9 5.8 All teachers should be required… 19 5.9 7.3 Certified teachers are more effective… 17 7.3 8.6 Non-certified teachers should pay all the costs… 13 5.3 5.6 School districts should pay… 7 4.8 5.4 Linking teacher salary to… achievement… 32 3.1 3.9 This school stresses excellence 34 3.7 3.4 I’m encouraged to make suggestions 18 2.9 4.2 When non-certified teachers are hired, school… 25 3.0 3.4 The salary bonus… large enough… motivate me 22 -1.5 -2.3 It is appropriate…bonuses to serve in urban… 23 -1.6 -2.4 It is appropriate…bonuses to serve in rural… 27 -1.2 -2.6 Improving my knowledge…enhances student learning 10 -2.2 -2.3 Differentiated compensation programs recognize teacher contributions… 26 -2.1 -3.6 A salary bonus motivates me to improve… 2 -3.6 -2.7 Differentiated compensation would not enhance the positive rel… 3 -2.7 -3.0 Differentiated compensation will have a negative impact… 1 -2.9 -3.5 Differentiated compensation helps recruit better-qualified… 15 -4.8 -3.6 A differentiated compensation… will result in a higher retention 4 -4.5 -4.6 Differentiated compensation helps retain teachers in critical shortage… 5 -5.2 -5.1 Differentiated compensation will positively affect teacher morale 6 -4.7 -5.3 Differentiated compensation provides incentives for growth 11 -7.6 -7.3 Students’ standardized test scores would improve… 20 -6.6 -6.4 Differentiated compensation programs help recruit teachers…improve…
Rating Scale Quality 42
Figure 8 Construct key map for teachers EXPECTED SCORE: MEAN (":" INDICATES HALF-SCORE POINT) (BY OBSERVED CATEGORY) -5 -4 -3 -2 -1 0 1 2 3 4 5 |-----+-----+-----+-----+-----+-----+-----+-----+-----+-----| NUM ITEM 1 1 : 2 : 3 : 4 33 There is too much peer pressure here to... 1 1 : 2 : 3 : 44 7 linking tchr salary to...achievement... 1 1 : 2 : 3 : 4 4 17 Non-certified tchrs should pay all the costs... 1 1 : 2 : 3 : 4 4 11 Stdnts' standardized test scores would improve... 1 1 : 2 : 3 : 4 4 2 DC would not enhance the positive rel... 1 1 : 2 : 3 : 4 4 18 When non-certified tchrs are hired, school... 1 1 : 2 : 3 : 4 4 5 DC will positively affect teacher morale 1 1 : 2 : 3 : 4 4 12 Relations between admin and instrctnl staff neg... 1 1 : 2 : 3 : 4 4 3 DC will have a negative impact... 1 1 : 2 : 3 : 4 4 21 It is appropriate...bonuses to teach in critical... 1 1 : 2 : 3 : 4 4 10 DC programs recgnze teacher contributions... 1 1 : 2 : 3 : 4 4 20 DC programs help recruit teachers...improve... 1 1 : 2 : 3 : 4 4 15 A DC...will result in a higher retention... 1 1 : 2 : 3 : 4 4 19 Certified teachers are more effective... 1 1 : 2 : 3 : 4 4 23 It is appropriate...bonuses to serve in rural... 1 1 : 2 : 3 : 4 4 25 The salary bonus...large enough...motivate me 1 1 : 2 : 3 : 4 4 22 It is appropriate...bonuses to serve in urban... 1 1 : 2 : 3 : 4 4 1 DC helps recruit better-qualified... 1 1 : 2 : 3 : 4 4 4 DC helps retain teachers in critical shortage... 1 1 : 2 : 3 : 4 4 26 A salary bonus motivates me to improve... 1 1 : 2 : 3 : 4 4 14 Career advancement opportunities...aligned... 1 1 : 2 : 3 : 4 4 8 Tchrs... will be less cooperative... 1 1 : 2 : 3 : 4 4 13 School districts should pay... 1 1 : 2 : 3 : 4 4 6 DC provides incentives for growth... 1 1 : 2 : 3 : 4 4 34 I'm encouraged to make suggestions... 1 1 : 2 : 3 : 4 4 24 If I receive a DC bonus I deserve it 1 1 : 2 : 3 : 4 4 32 This school stresses excellence 1 1 : 2 : 3 : 4 4 30 I feel a sense of ownership in this school 1 1 : 2 : 3 : 4 4 28 I identify with this school 1 1 : 2 : 3 : 4 4 16 All teachers should be required to be certif... 1 1 : 2 : 3 : 4 4 29 I take pride in being a part of this school 1 1 : 2 : 3 : 4 4 27 Improving my knowledge...enhances student learning 1 1 : 2 : 3 : 4 4 9 districts should support tchrs voluntarily... 1 1 : 2 : 3 : 4 4 31 I feel a sense of ownership in student learning |-----+-----+-----+-----+-----+-----+-----+-----+-----+-----| NUM ITEM -5 -4 -3 -2 -1 0 1 2 3 4 5 1113334332322212 111123901029261076641035222 PERSONS T S M S T
Rating Scale Quality 43
APPENDIX E: OUTPUT FOR MENTOR TEACHER-ACHIEVEMENT COACH Table 9 Summary statistics for mentor teacher/achievement coaches Measure Infit ZSTD Outfit ZSTD Person M 1.40 -.1 .0 S.D. .71 1.4 1.6 Item M .00 .0 .1 S.D. 1.34 1.5 1.6 Real person reliability .83 Real item reliability .96
Rating Scale Quality 44
Table 10 Item polarity for mentor teacher-achievement coaches Entry Number
Pt. Measure Correlation
Item
19 -.03 Certified teachers are more effective… 16 .00 All teachers should be required to be certified 33 .02 There is too much peer pressure here to… 31 .14 I feel a sense of ownership in student learning 9 .14 Districts should support teachers voluntarily… 7 .16 Linking teacher salary to…achievement… 32 .16 This school stresses excellence 30 .17 I feel a sense of ownership in this school 13 .17 School districts should pay… 34 .21 I’m encouraged to make suggestions… 27 .23 Improving my knowledge… enhances student learning 28 .23 I identify with this school 29 .24 I take pride in being a part of this school 8 .28 Tchrs…will be less cooperative… 26 .29 A salary bonus motivates me to improve… 14 .37 Career advancement opportunities…aligned… 25 .40 The salary bonus… large enough… motivate me 24 .43 If I receive a differentiated compensation bonus I deserve it 17 .44 Non-certified teachers should pay all the costs… 11 .47 Students’ standardized test scores would improve… 4 .52 Differentiated compensation helps retain teachers in critical shortage… 18 .54 When non-certified teachers are hired, school… 1 .54 Differentiated compensation helps recruit better-qualified… 21 .54 It is appropriate… bonuses to teach in critical… 2 .56 Differentiated compensation would not enhance the positive rel… 22 .57 It is appropriate… bonuses to serve in urban… 23 .58 It is appropriate…bonuses to serve in rural… 6 .59 Differentiated compensation provides incentives for growth… 12 .61 Relations between admin and instructional staff neg… 10 .62 Differentiated compensation programs recognize teacher contributions… 20 .66 Differentiated compensation programs help recruit teachers… improve 3 .69 Differentiated compensation will have a negative impact… 15 .69 A differentiated compensation… will result in a higher retention… 5 .77 Differentiated compensation will positively affect teacher morale
Rating Scale Quality 45
Figure 9 Empirical item-category measures for mentor teacher/achievement coaches -1 0 1 2 3 4 |-----------+-----------+-----------+-----------+-----------| NUM ITEM | 4 12 3 | 33 There is too much peer pressure here to... | 1 234 | 7 linking tchr salary to...achievement... | 1 2 34 | 17 Non-certified tchrs should pay all the costs... | 1 2 43 | 11 Stdnts' standardized test scores would improve... | 1 2 3 4 | 2 DC would not enhance the positive rel... | 12 3 4 | 21 It is appropriate...bonuses to teach in critical... | 1 23 4 | 18 When non-certified tchrs are hired, school... | 1 2 3 4 | 5 DC will positively affect teacher morale | 12 3 4 | 4 DC helps retain teachers in critical shortage... | 1 2 3 4 | 3 DC will have a negative impact... | 1 2 3 4 | 12 Relations between admin and instrctnl staff neg... | 12 3 4 | 23 It is appropriate...bonuses to serve in rural... | 1 2 3 4 | 15 A DC...will result in a higher retention... | 2 1 3 4 | 20 DC programs help recruit teachers...improve... | 12 3 4 | 22 It is appropriate...bonuses to serve in urban... | 1 2 3 4 | 1 DC helps recruit better-qualified... | 2 13 4 | 25 The salary bonus...large enough...motivate me | 1 2 3 4 | 10 DC programs recgnze teacher contributions... | 1 2 43 | 13 School districts should pay... | 2 3 4 | 14 Career advancement opportunities...aligned... | 1 2 3 4 | 6 DC provides incentives for growth... | 1 2 3 4 | 24 If I receive a DC bonus I deserve it | 1 23 4 | 26 A salary bonus motivates me to improve... | 2 3 4 | 8 Tchrs... will be less cooperative... | 3 4 2 | 19 Certified teachers are more effective... | 2 43 | 16 All teachers should be required to be certif... | 2 3 4 | 34 I'm encouraged to make suggestions... | 3 4 2 | 32 This school stresses excellence | 2 3 4 | 9 districts should support tchrs voluntarily... | 2 3 4 | 28 I identify with this school | 3 4 | 29 I take pride in being a part of this school | 3 4 | 27 Improving my knowledge...enhances student learning | 3 4 2 | 30 I feel a sense of ownership in this school | 3 4 2 | 31 I feel a sense of ownership in student learning |-----------+-----------+-----------+-----------+-----------| NUM ITEM -1 0 1 2 3 4 1 1 11111331931413 2133331 131 2 1 1 PERSONS T S M S T
Table 11 Category function for mentor teacher-achievement coaches Category Label Category Measure 1 -2.36 2 -.78 3 .65 4 2.57
Rating Scale Quality 46
Figure 10 Probability modes for mentor teacher-achievement coaches P ++---------+---------+---------+---------+---------+---------++ R 1.0 + + O | | B |1 | A | 111 4| B .8 + 111 444 + I | 11 444 | L | 11 44 | I | 11 44 | T .6 + 11 44 + Y | 11 44 | .5 + 1 33333333333 4 + O | 11 333 **3 | F .4 + *22222 33 44 33 + | 22222 11 3*222 44 333 | R | 222 *3 22 4 33 | E | 222 33 11 222 44 333 | S .2 + 222 33 11 44*22 333 + P | 22222 33 11 44 222 33| O |2 3333 44*11 2222 | N | 333333 444444 111111 2222222 | S .0 +*******444444444444444 1111111111111*********+ E ++---------+---------+---------+---------+---------+---------++ -3 -2 -1 0 1 2 3 PERSON [MINUS] ITEM MEASURE
Rating Scale Quality 47
Figure 11 V
ariable maps for mentor teacher-achievement coaches MAP OF PERSONS AND ITEMS MEASURE | MEASURE <more> --------------------- PERSONS -+- ITEMS --------------------- <rare> 5 + 5 | | | | | 4 + 4 | | | | X | 3 X + 3 T| X XX |T X X | XXX | X S| 2 XXXXX + X 2 XXXXX | XXXX | XXXX | XXXX M|S XXXXXX | XX 1 XXXXXXXXX + XX 1 XXXX | XX XXXXX S| XXX X | XXXX XX | X | X 0 X T+M X 0 | XXXX | X X | X | | -1 + -1 | X |S X | X | | -2 + XXX -2 | XXX | | |T | -3 + -3 | | | | | -4 + -4 <less> --------------------- PERSONS -+- ITEMS ------------------<frequent>
Table 12 Item misfit for mentor teacher-achievement coaches Entry Number
Infit ZSTD
Outfit ZSTD
Item
33 1.8 3.1 There is too much peer pressure here to… 16 2.5 2.8 All teachers should be required… 19 .5 2.3 Certified teachers are more effective… 12 -2.1 -2.3 Relations between admin and instructional staff negatively… 20 -2.3 -2.3 Differentiated compensation programs help recruit teachers…improve… 10 -2.3 -2.1 Differentiated compensation programs recognize teacher contributions… 3 -2.9 -2.7 Differentiated compensation will have a negative impact… 5 -3.6 -3.6 Differentiated compensation will positively affect teacher morale
Rating Scale Quality 48
Figure 12 Construct key map for mentor teacher/achievement coaches EXPECTED SCORE: MEAN (":" INDICATES HALF-SCORE POINT) (BY OBSERVED CATEGORY) -5 -3 -1 1 3 5 7 |---------+---------+---------+---------+---------+---------| NUM ITEM 1 1 : 2 : 3 : 4 4 33 There is too much peer pressure here to... 1 1 : 2 : 3 : 4 4 7 linking tchr salary to...achievement... 1 1 : 2 : 3 : 4 4 17 Non-certified tchrs should pay all the costs... 1 1 : 2 : 3 : 4 4 11 Stdnts' standardized test scores would improve... 1 1 : 2 : 3 : 4 4 2 DC would not enhance the positive rel... 1 1 : 2 : 3 : 4 4 21 It is appropriate...bonuses to teach in critical... 1 1 : 2 : 3 : 4 4 18 When non-certified tchrs are hired, school... 1 1 : 2 : 3 : 4 4 5 DC will positively affect teacher morale 1 1 : 2 : 3 : 4 4 4 DC helps retain teachers in critical shortage... 1 1 : 2 : 3 : 4 4 3 DC will have a negative impact... 1 1 : 2 : 3 : 4 4 12 Relations between admin and instrctnl staff neg... 1 1 : 2 : 3 : 4 4 23 It is appropriate...bonuses to serve in rural... 1 1 : 2 : 3 : 4 4 15 A DC...will result in a higher retention... 1 1 : 2 : 3 : 4 4 20 DC programs help recruit teachers...improve... 1 1 : 2 : 3 : 4 4 22 It is appropriate...bonuses to serve in urban... 1 1 : 2 : 3 : 4 4 1 DC helps recruit better-qualified... 1 1 : 2 : 3 : 4 4 25 The salary bonus...large enough...motivate me 1 1 : 2 : 3 : 4 4 10 DC programs recgnze teacher contributions... 1 1 : 2 : 3 : 4 4 13 School districts should pay... 1 1 : 2 : 3 : 4 4 14 Career advancement opportunities...aligned... 1 1 : 2 : 3 : 4 4 6 DC provides incentives for growth... 1 1 : 2 : 3 : 4 4 24 If I receive a DC bonus I deserve it 1 1 : 2 : 3 : 4 4 26 A salary bonus motivates me to improve... 1 1 : 2 : 3 : 4 4 8 Tchrs... will be less cooperative... 1 1 : 2 : 3 : 4 4 19 Certified teachers are more effective... 1 1 : 2 : 3 : 4 4 16 All teachers should be required to be certif... 1 1 : 2 : 3 : 4 4 34 I'm encouraged to make suggestions... 1 1 : 2 : 3 : 4 4 32 This school stresses excellence 1 1 : 2 : 3 : 4 4 9 districts should support tchrs voluntarily... 1 1 : 2 : 3 : 4 4 28 I identify with this school 1 1 : 2 : 3 : 4 4 29 I take pride in being a part of this school 1 1 : 2 : 3 : 4 4 27 Improving my knowledge...enhances student learning 1 1 : 2 : 3 : 4 4 30 I feel a sense of ownership in this school 1 1 : 2 : 3 : 4 4 31 I feel a sense of ownership in student learning |---------+---------+---------+---------+---------+---------| NUM ITEM -5 -3 -1 1 3 5 7 1 1 11136354385142 11 PERSONS T S M S T