delaware performance appraisal system second edition (dpas ii) · 2014. 8. 28. · 4. separate the...

Delaware Performance Appraisal System Second Edition

(DPAS II)

Year 2 Report

June 2009

Submitted By: Dr. Donald E. Beers Principal Investigator

2021-A North Halsted Street

Chicago, IL 60614 www.progresseducation.com

TABLE OF CONTENTS EXECUTIVE SUMMARY ................................................................................................ 1

Background................................................................................................................ 1 Summary of Results - Key Findings 2008-2009 ...................................................... 2

INTRODUCTION............................................................................................................. 4

RECOMMENDATIONS................................................................................................... 5

METHODS ...................................................................................................................... 8

RESULTS ..................................................................................................................... 12 Results - Q1.............................................................................................................. 12 Results - Q3.............................................................................................................. 15 Results – Q4, Q5, Q7, Q8, and Q9 .......................................................................... 18 Results – Q2, Q6, and Q12 ...................................................................................... 23 Results – Q13, Q14, Q17, Q18, Q20........................................................................ 25 Results – Q10 and 11............................................................................................... 31 Results – Q15........................................................................................................... 33 Results – Q16........................................................................................................... 33 Results – Q24........................................................................................................... 34 Results – Q25........................................................................................................... 36 Results – Q26........................................................................................................... 37 Results – Q22 and Q23............................................................................................ 45

APPENDIX A ................................................................................................................ 52 Pilot, Phase I, Other Statewide Comparison ......................................................... 52

Progress Education Corporation i DPAS II Report – June 2009

EXECUTIVE SUMMARY

Background The Delaware State Department of Education presented a very clear expectation for the evaluation of DPAS II. The stated goals of DPAS II are equally specific as stated on the Department of Education’s web site,

The purpose of DPAS II is two-fold:

• Quality assurance

• Professional growth Quality assurance focuses on the collection of credible evidence about the performance of educators. Evaluators use this evidence to make important decisions: recognizing effective practice, recommending continued employment, recommending an improvement plan, or beginning dismissal proceedings. Professional growth focuses on enhancing the skills and knowledge of educators. Through self-assessment and goal-setting, working with colleagues, taking courses, attending workshops, designing new programs, piloting new programs or approaches, developing proficiency in test data analysis, and many other learning opportunities, educators improve their professional practice in ways that will contribute to improved student learning. Both purposes serve accountability: to assure that educators are performing at an acceptable level and to provide professional growth opportunities that improve skills and knowledge.

The goal of this evaluation was to determine the reality of the current condition in meeting the stated goals. The majority of the findings center on the practices and processes of DPAS II. The practices provide an understanding of the quality of training, manuals, forms, and general deployment. The processes stem from fundamental policies and underlying theory about performance appraisal. This report is divided into four major sections: Introduction, Recommendations, Methods, and Results. Contained in these sections are the specific data collected and the methodologies used for analysis. The recommendations are very specific and tied to the major findings of the data collection process described under Results.

Progress Education Corporation 1 DPAS II Report – June 2009

Summary of Results - Key Findings 2008-2009 1) Among teachers, the items with the highest levels of desirable responses were:

a) that they are able to provide evidence of practice through discussion b) the five components used to evaluate performance is understandable c) the written feedback is aligned with the five components d) the feedback received is adequate

2) Among teachers, the items with the least desirable responses were: a) that classroom level DSTP provides an accurate picture of students’

progress b) that DSTP data helps adjust instruction for students c) additional training would make them more competent in the process d) that there was congruence with the results of school level data and

classroom level data. 3) Among specialists, the items with the highest levels of desirable responses were:

a) they are able to provide evidence of practice through discussion b) the evaluator completes paperwork in a reasonable time period c) the five components used to evaluate performance are understandable d) the evaluator handles the workload effectively e) the feedback received is adequate

4) Among specialists, the items with the least desirable responses were: a) that DSTP data gives an accurate picture of their school’s progress b) DSTP data helps them adjust goals for students and the school c) additional training would make them feel more competent in the process d) the evaluation system should continue in its current form.

5) Among administrators, the items with the highest level of desirable responses were:

a) The five components used to evaluate my performance are understandable

b) the Guide is easy to understand c) the Guide is helpful d) the training materials were helpful e) the five components used to evaluate performance are reasonable

6) Among administrators, the items with the least desirable responses were: a) DSTP gives an accurate picture of my school’s progress b) that the time it takes to complete the DPAS II paperwork is reasonable c) additional training would make them feel more competent in the process d) the current DPAS evaluation system should continue in its current form

7) The majority of teachers, specialists, and administrators gave the DPAS II system a grade of “B.” Additionally, when comparisons were made using the 2007-2008 sample, there were increases in the percent of teachers giving the process an “A” or “B.”



8) Results indicate that more teachers, specialists, and administrators selected the “Student Improvement” component as being a good indicator of performance (12 point increase among administrators). However, results on all the Data Related items show large percentages responding undesirably.

9) The majority of teachers and specialists stated they spent 0-5 hours on paperwork. The majority of administrators, however, spent more than 120 hours overall and more than 20 hours on paperwork.

10) About half of the teachers, specialists, and administrators did not believe that they needed additional training.

11) More administrators (99%) than teachers and specialists (approx. 75%) were familiar with the Department of Education website. Among all groups, the item “the short videos were helpful” received the least support.

12) When the new 2008/2009 districts were compared to Phase I districts, the Phase I districts had more positive results on 58 out of 60 items. Similarly, the pilot districts had more positive responses on 47 out of 60 items than the new 2008/2009 districts, although the differences were not as pronounced.

INTRODUCTION The purpose of the evaluation of DPAS II was to collect and compile data in order to make recommendations relating to the effectiveness and usability of the DPAS II process. 2008-2009 is the first year of statewide implementation for DPAS II. Progress Education Corporation was contracted by the Delaware Department of Education as a third-party evaluator to conduct all aspects of the evaluation. Upon receiving notification of being selected as the evaluator, the staff at Progress Education Corporation immediately began gathering contextual information, studying current manuals, and researching historical documents. Additionally, key staff members of the evaluation team visited the Delaware Department of Education to gain further insight into the DPAS II system and discuss any new expectations for the evaluation. Building upon the work that had already been done by the 1998 DPAS Revision Task Force and the DPAS II Advisory Committee, and following the evaluation questions as written in the DPAS II evaluation RFP, Progress Education Corporation developed and administered surveys, conducted interviews, and facilitated focus groups for teachers, specialists, administrators, and evaluators. All data collection forms (i.e. surveys, interview guides, and focus group questions) were created to provide ample information related to the DPAS II system. This included gathering qualitative and quantitative data on the criteria used in the DPAS II system; the forms for evaluating teachers, specialists and administrators; the manageability of the total system; the accuracy and reliability of the data being used in the system; usefulness of the training sessions and manuals; needed modifications; and the efficacy of the DPAS II program in achieving quality assurance and professional growth. More specifically, detailed survey, interview, and focus group items were generated to respond to 26 questions that were specified in the RFP.


RECOMMENDATIONS The recommendations for 2009 are categorized into four areas: student improvement; professional responsibilities; goal setting; and system implementation. Some recommendations repeat from previous studies. This may be due to the statewide implementation of DPAS II. Some issues identified during the pilot phase of the study resurfaced this year. A combination of the repeat recommendations and the improved understanding of the process with time, will improve the acceptance of DPAS II. This is clearly seen in Appendix A, the comparison of Pilot, Phase I and the remainder of the Delaware school districts. The student improvement component remains an issue for all groups. The interviews and focus groups remained positive this year yet still commented about a lack of understanding about the use of a “growth model”. Many indicated a lack of understanding about appropriate goals for student achievement. The recommendations remain largely the same this year in the face of the statewide implementation. Recommendations for student achievement include:

1. Improve access, understanding, and use of formative as well as summative data; 2. Provide district level training in the analysis and application of data; 3. Increase training on qualitative as well as quantitative data; 4. Create new videos demonstrating the establishment of goals for student

achievement; 5. Establish district level support for specialists and related arts teachers in

identifying appropriate data and use in establishing goals; 6. Facilitate the use of peer support groups to understand and apply student

achievement data to goals, and; 7. Establish a statewide data warehouse for the storage and access of student

achievement information. Professional responsibilities remain an area of emphasis in the qualitative and quantitative data. The survey demonstrates professional responsibilities section receives the lowest scores with regard to effective performance. Teachers and administrators showed a slight improvement over last year’s survey, but much work needs to be done. Interviews and focus groups surfaced a desire to create new opportunities for teacher leaders that focused on the expanding influence of experienced teachers. Recommendations for the professional responsibilities component include:

1. Facilitate the use of peer support groups to understand, set goals, and gather data in the professional responsibilities;

2. Create additional videos with examples of good and bad goal setting;


3. Place examples on the online forms for initial completion that can be removed for final printing;

4. Separate the professional growth from the communication expectations, and; 5. Create new opportunities for experienced teachers.

a. Teacher leadership roles such as committees or mentoring b. Value through DPAS II, department or grade level chair responsibilities

Goal setting and review across all components remains a concern. As last year, teachers, specialists, or administrators do not feel coerced when setting goals. All groups want additional help and examples in setting goals. Many respondents in the interviews and focus groups expressed concern that district or state level data is valued more than classroom data. The teachers and specialists in new districts to DPAS II expressed concern about the use of goals in the evaluation. Some indicate a worry that the failure to meet goals concerned them although they could not identify any cause for that concern other than a lack of understanding about the ramifications for not meeting a goal. This was not a great concern to teachers and specialists in the pilot and Phase I districts. Recommendations for goal setting include:

1. Facilitate the use of peer groups and learning communities to help individuals establish goals;

2. Evaluate the timelines for goal review; a. Is there room for more flexibility b. Can reviews be tied to data availability rather than timelines

3. Encourage the review of school, department, grade level goals prior to establishing individual goals;

4. Clarify the role of goals in the evaluation process, and; 5. Establish procedures for the review and updating of goals throughout the school

year. The recommendation in the area of system implementation is similar to last year. The qualitative data indicate much less concern and in fact, many respondents were complimentary of the training, website support, manual, and forms. The time to process DPAS II is still significant for first year participants. Knowing when things are due is a major issue for some. Recommendations for system implementation include:

1. Create a one page guideline sheet with all timelines; 2. Create an online user forum for teachers, specialists, and administrators where

groups can share ideas about data, goal setting, etc.; 3. Train administrators to view component five as ”action research”, and;



4. Create video with examples of what a reflective practice conference looks like and what it does not look like.

As last year, data availability remains a major challenge from the classroom to the statehouse. The ability to make timely decisions concerning instruction and budgets should not be hindered by a lack of time sensitive information about the growth of students. Teachers, specialists, and administrators recognize the need to collect information that is sensitive to the subtle changes in and needs of individual students. DPAS II is structured to make use of increasingly time sensitive data. Decisions informed by timely data will strongly support all groups governed by DPAS II.

METHODS Surveys, interview protocols, and focus group items were created for teachers, specialists, and administrators. Quantitative results were obtained via an on-line survey administered by K-12 Insight. The response rates for the teacher, specialist, and administrator surveys were 37%, 29%, and 43% respectively. Out of 8786 delivered teacher email invitations, 3268 teachers responded; out of 1085 specialists, 313 responded; and 194 out of 452 administrators responded. Qualitative information was obtained through interviews and focus groups. Three hundred twenty-one total interviews were conducted with teachers (n=257), specialists (n=28), and administrators (n=36). Six focus groups were conducted, three in the northern part of the state and three in the south. Two focus groups were conducted with teachers for a total of 12 participants. Two focus groups for administrators (n=15) and two for specialists (n=12) were conducted. For all groups (teachers, specialists, and administrators), the survey items were similar and followed the same pattern; however, some items were reworded specifically for each type of respondent. The first item of all the surveys assessed perceptions of each component of the DPAS II system–5 components for teachers, specialists, and administrators. These items were intended to gauge the participant’s perceptions of the criteria in each component. The 5 middle sections of the survey were made up of Likert items with a 4 point response scale ranging from Strongly Agree to Strongly Disagree. The Likert items were categorized into sections entitled: Evaluation Criteria, Documentation, Feedback, System Related Items, Data Related Items, and Department of Education website. The end of the survey consisted of a series of demographic questions. The 2008-2009 teacher results were subjected to a factor analysis to determine construct validity. Items were placed into constructs based on the highest factor loadings. Constructs were created if items loaded at a .4 factor level or higher; no item had a factor loading less than .5. There were 2 constructs that had items that formed separate constructs, however; the factor loadings were in the appropriate range to justify reporting them as one (for ease of interpretation). Reliability estimates were determined for each construct. With the exception of one construct, all reliability estimates were outstanding, at α=.8 or higher. The one exception was a construct with the following items: “The training was timely,” “Training in the process was adequate,” and “Additional training would make me feel more competent in the process.” As with 20072008, the first two items had adequate reliability estimates; the last item produced a low reliability estimate because there was great disparity among the respondents about whether additional training would make them feel more competent. The constructs and corresponding estimates are presented below:


Construct 1 α = .90

The five components used to evaluate my performance are understandable.

The five components used to evaluate my performance are reasonable indicators ofmy effectiveness. The criteria used to evaluate me for the planning and preparation component can be accurately judged by my evaluator. The criteria used to evaluate me for the classroom environment component can beaccurately judged by my evaluator. The criteria used to evaluate me for the instruction component can be accuratelyjudged by my evaluator. The criteria used to evaluate me for the professional responsibilities component can beaccurately judged by my evaluator. The criteria used to evaluate me for the student improvement component can beaccurately judged by my evaluator.

Applying all five components in my work is easy.

The written feedback I receive is aligned with the five components.

T he oral feedback I receive is aligned with the five components.


The forms play an important role in the overall evaluation.

I am able to provide the evidence and documentation needed by my evaluator for him/her to accurately determine my effectiveness.

I am able to provide evidence of my practice through artifact.

The time it takes to complete the DPAS II paperwork requirements is reasonable.

The forms are easy to complete.

I have access to the information I need to complete the forms.

The forms make the process easy to implement.

The information on the forms is consistent with determining the outcome of the evaluation.

The required paperwork is relevant to the evaluation.



My evaluator completes paperwork in a reasonable time period.

My evaluator handles the workload effectively.

Overall, the feedback I receive is adequate.

The oral feedback I receive is useful and applicable.

The written feedback I receive is useful and applicable.

In general, the conferences are valuable.

The forms completed after conferences are valuable.

I am able to provide evidence of my practice through discussion.

The timing of the conferences is good.

The number of conferences/conversations with my evaluator is adequate.


The system overall is easy to follow.

The evaluation process (observations, documentation, and conferences) provides adequate evidence of my teaching. The evaluation process (observations, documentation, and conferences) provides an accurate picture of my teaching.

The DPAS II system provides a better picture of my teaching versus the DPAS I system.

The Guide is helpful.

The Guide is easy to understand.

The evaluation did NOT interfere with my duties.

I perceive the system to be fair and equitable.

The DPAS evaluation system needs improving.

I believe the DPAS evaluation system works as intended.

I believe the current DPAS evaluation system should be continued in its current form.




Classroom level DSTP data gives me an accurate picture of my students' progress.

I was able to complete the data documentation requirements without difficulty.

There was enough training and/or support for me to accurately complete the forms related to student improvement.

DSTP data helps me adjust instruction for my students.

There was congruence with the results of school level data and my classroom data.

RESULTS

Results - Q1 1) Are the proposed criteria the best indicators of Effective Performance? Needs

Improvement Performance? Ineffective Performance? Of the 5 criteria in teacher evaluations, “Instruction” received the highest level of support for being a good indicator of performance. “Professional Responsibilities” was selected the least. Of the 5 criteria in specialist evaluations, “Professional Practice and Delivery of Service” was selected the most as being a good indicator of performance. “Student Improvement” was selected the least. Among administrators, the component selected the most for being a good indicator of performance was “Culture of Learning.” The component with least support from administrators was the “Professional Responsibilities” component. There was an increase from 2007-2008 to 2008-2009 in percent of administrators reporting that the “Student Improvement” component is a good indicator (58.8% versus 71.6% respectively).

Good Indicators of Performance – Teachers 08-09


Teachers Of the 5 major components (as defined in the DPAS II Guide) used in teacher evaluations, which do you believe are good

indicators of performance (check all that apply)?

Planning and Preparation

Classroom Environment Instruction Professional

Responsibilities Student

Improvement Did not answer Total

2007/2008 77.24% 80.06% 91.60% 44.03% 53.30% 1.18% 1274

2008/2009 73.90% 77.09% 90.28% 44.22% 59.31% 1% 3268

Note: Multiple answers per participant possible. Percentages added will not sum to 100 since a participant may select more thano ne answer for this question.

Good Indicators of Performance – Specialists 08-09

Specialists Of the 5 major components (as defined in the DPAS II Guide) used in specialist evaluations, which do you believe are

good indicators of performance?


Professional Practice and Delivery of

Service

Professional Collaboration

and Consultation

Professional Responsibilities

Student Improvement

Did not answer Total

2007/2008 70.73% 90.73% 76.10% 73.66% 42.93% 1.95% 205

2008/2009 68.05% 87.86% 69.01% 68.69% 47.92% 1% 313

Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select more than one answer for this question.


Good Indicators of Performance – Administrators 08-09

Administrator Of the 5 major components (as defined in the DPAS II Guide) used in administrator evaluations, which do you believe

are good indicators of performance? Vision and

Goals Culture of Learning Management Professional



2007/2008 70.59% 78.43% 74.51% 60.78% 58.82% 5.88% 51 2008/2009 68.04% 81.96% 81.44% 62.37% 71.65% 2% 194 Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select more thano ne answer for this question. Based on some comments during interviews, it appears that there is some confusion about the “Professional Responsibilities” expectations in this component among teachers. For example, one teacher stated:

“Professional Responsibilities examples make you feel guilty if you don't do them all - clarify these are examples not expectations.”

In other instances, teachers expressed that they didn’t understand how to show evidence of this component. As part of the interviews, questions were specifically asked about the “Student Improvement” component. The majority of interviewees had no issues with the “Student Improvement” component. The issues that did emerge dealt with: 1) for some teachers and specialists, this is not an appropriate component, 2) goal setting, 3) somehow changes in tests needs to be taken into account, and 4) data availability. Below are some selected statements from interviewees (in some cases these are paraphrased):


“This component is extremely difficult for teachers that do not teach core subjects, since there are not measures readily available for art, pe, music, etc.” “Specialist teachers are dependent upon the data from other teachers.” “Doesn't reflect the services provided as a specialist” “Timeline does not permit DSTP data to be used for summative.” “DSTP is a good way to measure goals, but results come back too late.” “DSTP results are not available until after summative evaluation deadline.” “Move summative to next year, data then available.” “Teachers set their own goals and those who set low goals meet them w/o difficulty, however, those who set high goals and don't meet them are placed on a student improvement plan.” “Not clear on whether setting goals correctly.” “Had difficulty setting goals. Process is not sensitive to the age of students.” “Requires extra work for sped teachers and is redundant to information that is developed for IEPs.”

There were conflicting statements with the use of classroom assessments:

“Need to use assessments that are classroom-based and objective.” “No issues - pleased that other classroom assessment can be used to judge growth.”

Results - Q3 3) Overall, is the system realistic? Among teachers, the highest rated item was “The five components used to evaluate my performance are understandable.” In 2007-2008, 92% of the teachers agreed or strongly agreed that the five components used to evaluate their performance are understandable. This percent increased to 95% in 2008-2009. The lowest rated item was “The criteria used to evaluate me for the student improvement component can be accurately judged by my evaluator.” Weighted score results (average of responses) are


also presented where Strongly Agree = 4, Agree = 3, Disagree = 2, and Strongly Disagree = 1. It is important to note that the weighted score on the item representing the student improvement component is 2.7 (not too far from 3, which represents the agree end of the scale). The percent who agreed or strongly agreed with this item is 64%. Six out of the 10 items in this survey section were in the positive side of the response scale.

Teachers Evaluation Criteria Items: Please indicate your level of agreement with each statement based on your belief

of being a good indicator of performance.

Strongly Agree Agree Disagree

Strongly Disagree Total

Weighted Score

The five components used to evaluate my performance are understandable. 23.57% 70.66% 5.21% 0.55% 3245 3.17 The written feedback I receive is aligned with the five components. 23.26% 67.39% 7.51% 1.85% 3195 3.12 The criteria used to evaluate me for the instruction component can be accurately judged by my evaluator. 21.04% 70.33% 7.49% 1.14% 3232 3.11 The oral feedback I receive is aligned with the five components. 22.86% 66.21% 8.87% 2.06% 3202 3.1 The criteria used to evaluate me for the classroom environment component can be accurately judged by my evaluator. 20.43% 68.48% 9.24% 1.85% 3236 3.07 The five components used to evaluate my performance are reasonable indicators of my effectiveness. 16.69% 68.52% 13.43% 1.36% 3247 3.01 The criteria used to evaluate me for the planning and preparation component can be accurately judged by my evaluator. 17.30% 64.93% 15.42% 2.34% 3242 2.97 The criteria used to evaluate me for the professional responsibilities component can be accurately judged by my evaluator. 16.40% 63.79% 17.33% 2.47% 3237 2.94 Applying all five components in my work is easy. 13.22% 54.12% 28.37% 4.29% 3215 2.76 The criteria used to evaluate me for the student improvement component can be accurately judged by my evaluator. 11.37% 52.31% 29.58% 6.75% 3229 2.68 As with the teachers, the highest rated item among the specialists was “The five components used to evaluate my performance are understandable.” However, the weight was somewhat lower among the specialists compared to teachers. The lowest rated item among specialists was “The criteria used to evaluate me for the student improvement component can be accurately judged by my evaluator.” However, it is important to note that over 56% agreed or strongly agreed with the item. The weighted score is reduced because of the number of respondents in the strongly disagree column. Of the 10 items in this section, only 1 had a weighted score on the desirable end of the response scale.


Specialists Evaluation Criteria Items: Please indicate your level of agreement with each statement based on your belief




Weighted Score

The five components used to evaluate my performance are understandable. 15.97% 74.44% 7.67% 1.28% 313 3.06 The written feedback I receive is aligned with the five components. 15.65% 66.45% 12.14% 2.56% 313 2.98 The oral feedback I receive is aligned with the five components. 15.34% 67.09% 13.10% 2.56% 313 2.97 The criteria used to evaluate me for the professional practice and delivery of service component can be accurately judged by my evaluator. 13.10% 66.77% 15.97% 3.51% 313 2.9 The criteria used to evaluate me for the planning and preparation component can be accurately judged by my evaluator. 12.46% 66.45% 16.61% 3.51% 313 2.89 The criteria used to evaluate me for the professional responsibilities component can be accurately judged by my evaluator. 11.82% 68.69% 15.34% 3.83% 313 2.89 The five components used to evaluate my performance are reasonable indicators of my effectiveness. 10.54% 69.97% 16.29% 3.19% 313 2.88 The criteria used to evaluate me for the professional collaboration and consultation component can be accurately judged by my evaluator. 10.22% 65.50% 20.45% 3.83% 313 2.82 Applying all five components in my work is easy. 7.67% 52.08% 32.27% 7.03% 313 2.61 The criteria used to evaluate me for the student improvement component can be accurately judged by my evaluator. 7.67% 47.92% 32.59% 10.86% 313 2.53

Administrators also rated the item “The five components used to evaluate my performance are understandable” positively. When the weighted score is compared among the items, 6 of the 9 items have scores on the positive end of the response scale. The item affiliated with Student Improvement, “I agreed with the goals that were set for me under the Student Improvement component,” received positive responses among administrators (72% agreed and 18% strongly agreed). In 2007-2008, for the item “applying all five components in my work is easy,” about half of the administrators responded on the “agree” end of the scale (51%) and about half responded on the “disagree” end of the scale (49%). In 2008-2009, the percent that responded on the “agree” end of the scale increased to 69%.


Administrators Evaluation Criteria Items: Please indicate your level of agreement with each statement based on your belief




Weighted Score

The five components used to evaluate my performance are understandable. 27.84% 65.46% 6.70% 0% 194 3.21 The five components used to evaluate my performance are reasonable indicators of my effectiveness. 20.10% 69.59% 10.31% 0% 194 3.1 The oral feedback I receive is aligned with the five components. 23.28% 62.96% 10.58% 3.17% 189 3.06 I agreed with the goals that were set for me under the Student Improvement component. 18.09% 71.81% 7.45% 2.66% 188 3.05 The written feedback I receive is aligned with the five components. 21.62% 63.24% 13.51% 1.62% 185 3.05 My evaluator was able to accurately judge my performance in the Vision and Goals component. 21.05% 63.68% 11.05% 4.21% 190 3.02 The criteria used to evaluate me in the Student Improvement component can be accurately judged by my evaluator. 14.74% 59.47% 22.11% 3.68% 190 2.85 Applying all five components in my work is easy. 16.23% 52.88% 29.84% 1.05% 191 2.84 The survey used to evaluate me on the Delaware Administrator standards provide an accurate picture of my effectiveness. 8.99% 51.32% 32.80% 6.88% 189 2.62

Results – Q4, Q5, Q7, Q8, and Q9 4) How much time does it take for the person being evaluated to complete the

required paperwork? 5) How much time does it take for the evaluator to complete the required

paperwork? 7) Can the evaluators handle the workload of the evaluations? 8) Are the forms understandable and useable? 9) Do the forms provide the appropriate data for the evaluator to fairly and

accurately assess an individual’s performance?

Documentation: Teachers Please indicate your level of agreement with each of the following statements.



Weighted Score

My evaluator completes paperwork in a reasonable time period. 28.41% 57.97% 9.31% 4.31% 3224 3.1 I am able to provide the evidence and documentation needed by my evaluator for him/her to accurately determine my effectiveness. 18.87% 71.25% 9.11% 0.77% 3228 3.08 My evaluator handles the workload effectively. 26.53% 59.13% 10.16% 4.19% 3200 3.08


I am able to provide evidence of my practice through artifact. 16.44% 71.62% 11.07% 0.87% 3217 3.04 I have access to the information I need to complete the forms. 14.31% 76.17% 8.50% 1.02% 3236 3.04

The information on the forms is consistent with determining the outcome of the evaluation. 9.57% 71.42% 16.62% 2.39% 3219 2.88 The required paperwork is relevant to the evaluation. 9.58% 65.15% 21.25% 4.01% 3214 2.8 The forms play an important role in the overall evaluation. 8.95% 62.87% 25.46% 2.72% 3127 2.78 The forms are easy to complete. 7.84% 65.23% 23.65% 3.28% 3201 2.78 The forms make the process easy to implement. 7.87% 62.71% 25.17% 4.25% 3202 2.74 The time it takes to complete the DPAS II paperwork requirements is reasonable. 6.88% 58.13% 26.94% 8.05% 3241 2.64

Teachers - On an annual basis, how much time do you spend on paperwork relating to the DPAS II system? The highest level of positive responses from teachers was on the items relating to their evaluator and the evidence needed as documentation for the components. The items relating to forms had fewer positive responses. When asked to select the category that fits best regarding the time spent on paperwork, the majority of teachers spent 0-5 hours on paperwork relating to the DPAS II system. The next highest category selected was 6-10 hours.


Documentation: Specialists Please indicate your level of agreement with each of the following statements.



Weighted Score

The evaluator completes paperwork in a reasonable time period. 21.90% 66.99% 7.52% 3.59% 306 3.07 My evaluator(s) handle the workload effectively. 22.11% 65.02% 9.24% 3.63% 303 3.06 I have access to the information I need to complete the forms. 10.97% 77.10% 10.32% 1.61% 310 2.97 I am able to provide evidence of my practice through artifact. 15.31% 66.45% 15.96% 2.28% 307 2.95 I am able to provide the evidence and documentation needed by my evaluator for him/her to accurately determine my effectiveness. 14.10% 66.99% 16.67% 2.24% 312 2.93 The information on the forms is consistent with determining the outcome of the evaluation. 7.82% 66.45% 21.50% 4.23% 307 2.78 The forms play an important role in the overall evaluation. 7.97% 63.46% 24.25% 4.32% 301 2.75

The forms are easy to complete. 8.14% 61.89% 25.73% 4.23% 307 2.74 The required paperwork is relevant to the evaluation. 6.80% 63.11% 24.60% 5.50% 309 2.71 The forms make the process easy to implement. 7.19% 59.80% 28.10% 4.90% 306 2.69 The time it takes to complete the DPAS II paperwork requirements is reasonable. 6.75% 60.45% 27.33% 5.47% 311 2.68

Specialists - On an annual basis, how much time do you spend on paperwork relating to the DPAS II system?


The majority of specialists responded positively to all items related to documentation. However, there are between 25 – 33% of specialists who responded on the undesirable end of the response scale on 6 of the 11 items. All 6 of those items related to the forms and paperwork in some manner. The majority of specialists believe that their evaluator completes paperwork in a reasonable time period and that their evaluator handles the workload effectively. The majority of specialists also believe that they are able to provide the needed evidence and they have access to the information. Similar to the teachers, the majority of specialists responded that they spent 5 hours or less on the paperwork relating to the DPAS II system. The next highest category selected among specialists was 6-10 hours.

Documentation: Administrators Please indicate your level of agreement with each of the following statements.



Weighted Score

I have access to the information I need to complete the forms. 13.54% 80.21% 5.73% 0.52% 192 3.07 My evaluator(s) handle the workload effectively. 26.06% 59.57% 10.11% 4.26% 188 3.07 I am able to provide the evidence and documentation needed by my evaluator for him/her to accurately determine my effectiveness. 16.15% 72.92% 9.38% 1.56% 192 3.04 The evaluator completes paperwork in a reasonable time period. 22.46% 63.10% 10.70% 3.74% 187 3.04 The forms play an important role in the overall evaluation. 12.11% 68.95% 18.42% 0.53% 190 2.93 The information on the forms is consistent with determining the outcome of the evaluation. 10.00% 74.74% 13.68% 1.58% 190 2.93 The required paperwork is relevant to the evaluation. 11.05% 68.42% 18.95% 1.58% 190 2.89

The forms are easy to complete. 7.81% 66.15% 21.88% 4.17% 192 2.78 The forms make the process easy to implement. 7.29% 63.02% 26.56% 3.13% 192 2.74 The time it takes to complete the DPAS II paperwork requirements is reasonable. 6.22% 49.74% 33.16% 10.88% 193 2.51


Administrators - On an annual basis, how many hours overall do you spend on DPAS II?

Administrators - On an annual basis, how many hours do you spend on paperwork relating to the DPAS II system?


Administrators - On an annual basis, how many hours do you spend on paperwork relating to the administrative portion of DPAS II?

The highest level of positive responses from administrators was on the items relating to their evaluator and the evidence needed as documentation for the components. The weighted score drops into the undesirable end of the scale on all items relating to the forms and paperwork. The item with the most responses disagree/strongly disagree (44%) was “The time it takes to complete the DPAS II paperwork requirements is reasonable.” In 2007-2008, the results indicated that 41% of administrators spent 11-15 hours on the administrative portion of the evaluation process; this increased to 50% in 2008-2009. In 2007-2008, close to 40% spent more than 120 hours overall; this increased slightly in 2008-2009 to 44%.

Results – Q2, Q6, and Q12 2) Do the number of observations and other collections of evidence provide enough

information for an evaluator to make an accurate assessment of performance? 6) Is there an appropriate balance between conversation or conferencing and

documentation? 12) Are the conferences meaningful and timely?


Teachers Feedback



Weighted Score

I am able to provide evidence of my practice through discussion. 24.08% 70.44% 4.48% 1.00% 3214 3.18 Overall, the feedback I receive is adequate. 23.04% 67.82% 7.03% 2.11% 3216 3.12 The oral feedback I receive is useful and applicable. 24.61% 64.37% 8.81% 2.20% 3222 3.11 The written feedback I receive is useful and applicable. 22.75% 65.11% 9.84% 2.30% 3213 3.08 In general, the conferences are valuable. 23.06% 62.77% 11.72% 2.45% 3218 3.06

The number of conferences/conversations with my evaluator is adequate. 20.32% 66.08% 10.47% 3.14% 3219 3.04 The timing of the conferences is good. 18.85% 67.72% 10.88% 2.56% 3209 3.03

The forms completed after conferences are valuable. 14.72% 61.16% 21.10% 3.02% 3180 2.88 A majority of teachers responded on the positive end of the scale for all items related to feedback. However, in 2007-2008 and 2008-2009, over 20% (23% and 24% respectively) were on the disagree/strongly disagree end of the scale in 2007-2008 on the item “The forms completed after conferences are valuable.”

Specialists Feedback



Weighted Score

I am able to provide evidence of my practice through discussion. 22.40% 72.73% 3.25% 1.62% 308 3.16

Overall, the feedback I receive is adequate. 19.48% 69.81% 7.47% 3.25% 308 3.06 The written feedback I receive is useful and applicable. 19.60% 68.44% 8.31% 3.65% 301 3.04 The oral feedback I receive is useful and applicable. 19.61% 66.99% 9.15% 4.25% 306 3.02

The number of conferences/conversations with my evaluator is adequate. 19.02% 66.89% 11.48% 2.62% 305 3.02 In general, the conferences are valuable. 17.38% 68.52% 10.49% 3.61% 305 3 The timing of the conferences is good. 15.95% 69.44% 11.30% 3.32% 301 2.98

The forms completed after conferences are valuable. 8.72% 66.11% 20.47% 4.70% 298 2.79 Similar to 2007-2008, among specialists, the highest and lowest mean scores were on the items “I am able to provide evidence of my practice through discussion” and “The forms completed after conferences are valuable,” respectively. The item relating to forms after the conferences had 26% of specialists respond on the disagree/strongly disagree end of the scale.


Administrators Feedback



Weighted Score

The oral feedback I receive is useful and applicable. 27.51% 56.61% 13.23% 2.65% 189 3.09 The timing of conferences is good. 20.32% 63.10% 12.83% 3.74% 187 3

The number of conferences/conversations with my evaluator is adequate. 22.22% 59.79% 12.70% 5.29% 189 2.99 Overall, the feedback I receive is adequate. 19.68% 62.23% 14.36% 3.72% 188 2.98 The written feedback I receive is useful and applicable. 18.38% 61.62% 16.22% 3.78% 185 2.95 Similar to 2007-2008, among administrators, the oral feedback item received the most positive responses. The majority of administrators responded positively to the items related to feedback. The lowest scoring item among administrators was “The written feedback I receive is useful and applicable.”

Results – Q13, Q14, Q17, Q18, Q20 13) Does the proposed system demonstrate equity among Teachers? Specialists?

Administrators? 14) Are educators’ ratings, under the DPAS II, reasonably aligned with prior

evaluations under DPAS I? 17) Is the training adequate? 18) Is the Guide useful? 20) Are the content, materials, timelines, and delivery methods appropriate and

effective? Teachers

System Related Items



Weighted Score

The Guide is helpful. 11.04% 73.53% 13.68% 1.76% 3188 2.94 The evaluation did NOT interfere with my duties. 13.19% 68.69% 14.47% 3.66% 3200 2.91 I perceive the system to be fair and equitable. 10.13% 72.51% 14.38% 2.97% 3198 2.9 The Guide is easy to understand. 9.92% 69.07% 18.93% 2.07% 3185 2.87 The evaluation process (observations, documentation, and conferences) provides adequate evidence of my teaching. 9.78% 67.43% 19.71% 3.07% 3221 2.84 The system overall is easy to follow. 8.90% 68.32% 19.88% 2.90% 3204 2.83 The evaluation process (observations, documentation, and conferences) provides an accurate picture of my teaching. 9.04% 64.62% 23.14% 3.20% 3219 2.79


The DPAS II system provides a better picture of my teaching versus the DPAS I system. 8.13% 59.04% 28.51% 4.32% 3076 2.71

How often do you use or refer to the Guide for DPAS II? Teachers

Responses Count % Percentage of total respondents Never 320 9.81% 1 time per year 620 19.01% 2-3 times per year 1526 46.80% 4-5 times per year 539 16.53% 6 or more times per year 242 7.42% (Did not answer) 14 0.43% Total Responses 3261 20% 40% 60% 80% 100% The majority of teachers responded “Agree” to all items related to the system overall. However, when the disagree/strongly disagree responses are taken into account, the weighted means are all below the desirable end of the scale. The item with the highest mean among the system related items was “The Guide is helpful.” As was the case in 2007-2008, the item with the lowest mean was “The DPAS II system provides a better picture of my teaching versus the DPAS I system.” When asked how often teachers refer to the Guide, 47% selected “2-3 times per year.” Ten percent responded “Never.”

Specialists System Related Items



Weighted Score

The Guide is helpful. 10.16% 75.08% 11.48% 3.28% 305 2.92 The evaluation did NOT interfere with my duties. 12.96% 69.77% 13.95% 3.32% 301 2.92 The Guide is easy to understand. 9.21% 72.37% 16.12% 2.30% 304 2.88 I perceive the system to be fair and equitable. 8.61% 72.19% 14.90% 4.30% 302 2.85 The system overall is easy to follow. 7.87% 65.25% 21.97% 4.92% 305 2.76 The evaluation process (observations, documentation, and conferences) provides adequate evidence of my performance. 6.21% 66.34% 21.24% 6.21% 306 2.73 The DPAS II system provides a better picture of my performance versus the DPAS I system. 6.94% 58.33% 30.56% 4.17% 288 2.68 The evaluation process (observations, documentation, and conferences) provides an accurate picture of my performance. 5.88% 59.80% 27.78% 6.54% 306 2.65


How often do you use or refer to the Guide for DPAS II? Specialists

Responses Count % Percentage of total respondents Never 46 14.70% 1 time per year 66 21.09% 2-3 times per year 141 45.05% 4-5 times per year 46 14.70% 6 or more times per year 9 2.88% (Did not answer) 5 1.60% Total Responses 313 20% 40% 60% 80% 100% Among specialists, there were 2 system related items that had the highest mean score: 1) “The Guide is helpful,” and 2) “The evaluation did NOT interfere with my duties.” The item with the lowest weighted mean score was “The evaluation process (observations, documentation, and conferences) provides an accurate picture of my performance.” The majority of specialists reported that they refer to the Guide “2-3 times per year.” The next highest category selected was “1 time per year.”



Weighted Score

The Guide is helpful. 25.52% 69.27% 3.65% 1.04% 192 3.2 The Guide is easy to understand. 22.11% 68.95% 7.89% 0.53% 190 3.13 I perceive the system to be fair and equitable. 9.04% 73.40% 15.43% 1.60% 188 2.9 The system overall is easy to follow. 9.79% 69.59% 17.01% 3.09% 194 2.87 The evaluation did NOT interfere with my duties. 11.58% 66.32% 17.89% 3.68% 190 2.86 The evaluation process provides adequate evidence of my performance. 8.90% 69.63% 17.80% 3.14% 191 2.85 The evaluation process provides an accurate picture of my performance. 7.85% 63.35% 25.13% 3.14% 191 2.76 The DPAS II system provides a better picture of my performance versus the DPAS I system. 10.16% 56.15% 27.81% 5.35% 187 2.72


How often do you refer to the guide for DPAS II? Administrators

Responses Count % Percentage of total respondents Never 0 0% 1 time per year 2 1.03% 2-3 times per year 40 20.62% 4-5 times per year 42 21.65% 6 or more times per year 109 56.19% (Did not answer) 0 0% Total Responses 193 20% 40% 60% 80% 100% Among administrators, and similar to 2007-2008, “The Guide is helpful” received the most positive responses. Conversely, the lowest weighted mean score corresponds to “The DPAS II system provides a better picture of my performance versus the DPAS I system.” Overall, the majority of administrators responded positively on all the items in this section. When asked “How often do you refer to the guide for DPAS II,” the category with the most responses from administrators was 6 or more times per year. No administrators selected “never” as the response to this item.

Teachers Training Related Items



Weighted Score

The training was timely. 9.10% 69.19% 18.44% 3.27% 3210 2.84 Training in the process is adequate. 8.17% 65.36% 21.89% 4.58% 3207 2.77 Additional training would make me feel more competent in the process. 9.41% 39.84% 43.70% 7.04% 3208 2.52 Teachers

From the following list, select the components of the DPAS process where you need additional training. (check all that apply)

None

Component 1 - Planning

and Preparation

Component 2 - Professional Practice and Delivery of

Service

Component 3 -Professional

Collaboration and

Consultation


Responsibilities

Component 5 - Student

Improvement


2007/2008 48.43% 5.18% 7.38% 13.42% 8.48% 25.51% 12.72% 1274

2008/2009 53.17% 11.41% 14.38% 21.34% 13.71% 26.43% 5.00% 3261

Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select more thano ne answer for this question.


Teachers From the following list, select specific aspects of the DPAS process where you need additional training. (Check all

that apply)

Providing evidence of work

Completing paperwork

Interpreting data

Presenting data

Managing the

requirements of the

evaluation with my regular duties

Understanding the Guide

Preparing for

conferences


2007/2008 15.38% 16.72% 28.18% 21.90% 21.04% 16.41% 10.05% 37.99% 1274

2008/2009 18.89% 18.34% 28.15% 21.22% 25.76% 14.41% 12.60% 36.98% 3261

Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select more than o ne answer for this question. Of the training items, among teachers, the lowest mean score was related to whether additional training would make teachers feel more competent in the process—49% responded on the “Agree/Strongly Agree” end of the scale and 51% responded on the “Disagree/Strongly Disagree” end of the scale; this was the exact opposite in 2007-2008. For both items relating to specific topics for additional training, the majority of teachers either did not respond or felt they did not need additional training. The next highest categories were related to data and/or the student improvement component, and managing the requirements in conjunction with regular duties. Of note, there were increases in the percent responding they need training in “Professional Practice and Delivery of Service” and “Professional Collaboration and Consultation,” when compared to 2007-2008.

Specialists Training Related Items



Weighted Score

The training for the districts was timely. 10.06% 67.53% 19.16% 3.25% 308 2.84 Training in the process is adequate. 7.77% 63.11% 25.24% 3.88% 309 2.75 Additional training would make me feel more competent in the process. 8.14% 42.02% 42.67% 7.17% 307 2.51

Specialists From the following list, select the components of the DPAS process where you need additional training.

None Component 1 - Planning and Preparation

Component 2 - Professional Practice and

Delivery of Service

Component 3 - Professional

Collaboration and Consultation


Responsibilities


Improvement


2007/2008 46.34% 6.34% 6.34% 5.37% 3.90% 28.29% 19.02% 205

2008/2009 53.35% 14.06% 11.82% 18.85% 12.78% 30.67% 8.63% 313

Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select more than onea nswer for this question.


Specialists From the following list, select the components of the DPAS process where you need additional training.



Interpreting data

Presenting data

Managing the

requirements of the



Preparing for

conferences


2007/2008 17.56% 20.00% 29.27% 24.88% 22.44% 15.61% 9.27% 40.00% 205

2008/2009 20.45% 16.93% 25.24% 20.13% 22.36% 15.65% 12.46% 42.81% 313

Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select more thano ne answer for this question. Among specialists, 50% of the respondents were on both ends of the scale on the item “Additional training would make me feel more competent in the process.” As with the teachers, the largest percent of specialists either did not respond or answered “none” when asked to indicate the areas in which they need additional training. The next largest percent of respondents checked the data related categories and/or the student improvement component, and managing the requirements of the duties in conjunction with their regular duties. In all DPAS II components for the specialists, there were increases in the percent responding they needed additional training when compared to 2007-2008.

Administrators Training Related Items



Weighted Score

The training for the districts was timely. 19.69% 68.91% 10.36% 0.52% 193 3.08 Training in the process is adequate. 17.62% 67.88% 13.47% 0.52% 193 3.03 Additional training would make me feel more competent in the process. 10.36% 37.82% 47.15% 4.15% 193 2.55

Administrators From the following list, select the components of the DPAS process where you need additional training.

Component 1 -

Vision and Goals

Component 2 - Culture of Learning

Component 3 - Management


Responsibilities



2007/2008 17.65% 19.61% 9.80% 7.84% 39.22% 39.22% 51

2008/2009 14.95% 16.49% 11.86% 9.79% 29.38% 53.09% 194

Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select more than onea nswer for this question.


Administrators From the following list, select the components of the DPAS process where you need additional training.



Interpreting data

Presenting data

Managing the

requirements of the



Preparing for

conferences


2007/2008 13.73% 7.84% 33.33% 21.57% 19.61% 1.96% 15.69% 37.25% 51

2008/2009 18.56% 12.37% 18.04% 17.53% 24.23% 5.67% 19.59% 44.33% 194

Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select more thano ne answer for this question. Among administrators, 48% responded on the “Agree/Strongly Agree” end of the scale on the item “Additional training would make me feel more competent in the process.” For administrators, when asked what components or areas do they need additional training, the majority did not answer. There were decreases in the percent selecting the items “Presenting data” and “Interpreting data. There were increases in percent selecting the following categories from 2007-2008 to 2008-2009: “Providing evidence of work,” “Completing paperwork,” “Managing the requirements of the evaluation with my regular duties,” “Understanding the Guide,” and “Preparing for conferences.

Results – Q10 and 11 10) What specific issues were encountered with Component V of the teacher and

specialist processes? 11) What was the outcome when using classroom level DSTP data versus school

level DSTP data?

Teachers Data Related Items



Weighted Score

I was able to complete the data documentation requirements without difficulty. 4.29% 60.05% 31.27% 4.39% 3099 2.64

There was enough training and/or support for me to accurately complete the forms related to student improvement. 4.46% 58.84% 31.79% 4.91% 3095 2.63

There was congruence with the results of school level data and my classroom data. 3.52% 60.06% 29.70% 6.73% 2987 2.6 DSTP data helps me adjust instruction for my students. 3.57% 43.45% 38.41% 14.56% 3077 2.36


Classroom level DSTP data gives me an accurate picture of my students' progress. 2.99% 32.90% 46.77% 17.34% 3109 2.22

Specialists Data Related Items



Weighted Score

I was able to complete the data documentation requirements without difficulty. 7.67% 56.45% 30.31% 5.57% 287 2.66 There was enough training and/or support for me to accurately complete the forms related to student improvement. 7.88% 54.79% 32.19% 5.14% 292 2.65

DSTP data helps me adjust goals for my school and/or students. 4.05% 41.22% 38.18% 16.55% 296 2.33 DSTP data gives an accurate picture of my school's progress. 2.68% 32.11% 45.48% 19.73% 299 2.18

Administrators Data Related Items



Weighted Score

There was enough training and/or support for me to accurately complete the forms related to data. 10.75% 69.35% 19.35% 0% 186 2.91

I was able to complete the data documentation requirements without difficulty. 5.91% 71.51% 19.89% 2.15% 186 2.82 DSTP data helps me adjust goals for my school. 8.42% 66.84% 17.89% 6.32% 190 2.78 DSTP data gives an accurate picture of my school's progress. 3.17% 32.80% 49.74% 13.76% 189 2.26 Among teachers and specialists, the item with the highest mean in the data construct was “I was able to complete the data documentation requirements without difficulty.” Among teachers, only 36% responded strongly agree/agree on “Classroom level DSTP data gives me an accurate picture of my students' progress;” thus corresponding with the lowest weighted score among all the items. Similarly, among specialists and administrators, the lowest weighted score was “DSTP data gives an accurate picture of my school's progress.” For administrators, the item with the highest mean score among the data related items was “There was enough training and/or support for me to accurately complete the forms related to data.”


Results – Q15 15) Are there differences in how the DPAS II works for novice and experienced

educators? If so, what are the differences? Using the variable “total years experience” for teachers, analyses were performed to determine whether any differences existed on the survey items based on level of experience. The teacher experience data were disaggregated into quartile categories. On every item, the results revealed slightly more positive perceptions for those who had fewer years experience. This was also the case with the item that asked the respondents to give the process a grade. Higher percentages of those with fewer years of experience gave the process a grade of A or B.

Results – Q16 16) Is the “Improvement Plan” process helpful? There were 43 teacher respondents who indicated they were on improvement plans. There were 2 specialists and 5 administrators. Subsequently, only the teacher responses to the improvement plan items are presented. Among teachers on improvement plans, 70% responded “Strongly Agree” or “Agree” when asked if the improvement plan outlined measurable goals to work toward achieving. Slightly fewer than 50% responded on the strongly agree/agree end of the scale for “There are adequate resources to implement improvement plans.” Teachers

Were you placed on an improvement plan this year? Yes No Total2008/2009 1.32% 98.68% 3261

Teachers Improvement Plans



Weighted Score

The Improvement Plan outlined measurable goals for me to work toward achieving. 4.65% 65.12% 16.28% 13.95% 43 2.6 The Improvement Plan recommendations were useful. 4.65% 53.49% 18.60% 23.26% 43 2.4 The Improvement Plan process helped direct my professional growth goals. 4.65% 51.16% 18.60% 25.58% 43 2.35 There are adequate resources to implement improvement plans. 2.38% 45.24% 33.33% 19.05% 42 2.31


Results – Q24 24) Does the system provide the necessary support and resources to allow

educators to reflect on and identify ways to improve their practice?

Teachers - Are you familiar with the Department of Education website that supports DPAS II?

Teachers Website Evaluation: Please indicate your level of agreement with the following statements.



Weighted Score

The online manual was useful. 9.28% 78.46% 11.14% 1.13% 2307 2.96 The training materials were helpful. 7.53% 80.97% 10.29% 1.20% 2244 2.95 The online manual was easy to use. 9.08% 76.82% 12.92% 1.18% 2291 2.94 The FAQs addressed my questions. 7.41% 77.76% 13.82% 1.00% 2199 2.92 The website provides me with all the information I need on DPAS II. 8.37% 74.89% 15.50% 1.24% 2258 2.9 The short videos were helpful. 6.80% 71.06% 20.53% 1.62% 2163 2.83


Specialists - Are you familiar with the Department of Education website that supports DPAS II?

Specialists Website Evaluation: Please indicate your level of agreement with the following statements.



Weighted Score

The online manual was easy to use. 9.48% 78.67% 10.43% 1.42% 211 2.96 The online manual was useful. 7.55% 79.72% 10.38% 2.36% 212 2.92 The training materials were helpful. 8.21% 73.91% 15.94% 1.93% 207 2.88 The website provides me with all the information I need on DPAS II. 8.57% 72.86% 16.19% 2.38% 210 2.88 The FAQs addressed my questions. 6.25% 72.60% 19.23% 1.92% 208 2.83 The short videos were helpful. 7.61% 65.99% 22.34% 4.06% 197 2.77


Administrators - Are you familiar with the Department of Education website that supports DPAS II?

Administrators Website Evaluation: Please indicate your level of agreement with the following statements.



Weighted Score

The training materials were helpful. 16.39% 78.69% 4.37% 0.55% 183 3.11 The online manual was easy to use. 15.34% 78.41% 5.68% 0.57% 176 3.09 The online manual was useful. 13.97% 81.01% 4.47% 0.56% 179 3.08 The FAQs addressed my questions. 15.88% 77.06% 5.88% 1.18% 170 3.08

The website provides me with all the information I need on DPAS II. 14.75% 77.05% 7.65% 0.55% 183 3.06

The short videos were helpful. 16.57% 72.19% 10.65% 0.59% 169 3.05 Almost all administrative respondents were aware of the DPAS II website. Fewer teachers and specialists were aware of it. Among administrators, there were mostly positive responses on all items relating to the website (training materials, online manual, FAQs, videos). Among teachers and specialists, there were 21 – 26% respectively who disagreed or strongly disagreed that the short videos were helpful.

Results – Q25 25) What unique circumstances were encountered? How were they handled? No specific unique circumstances were brought to the attention of the interviewers or during the focus groups.


Results – Q26 26) As a whole, how did the system work? Teachers, specialists, and administrators were asked to give the evaluation process a grade (A – F) and to indicate their level of agreement with 3 general items about the system. For the first analysis, comparisons were made using the sample from 2007/2008 and comparing the results to the same sample in 2008/2009 for teachers. A larger percent gave the process a grade of A and B in 2008/2009, indicating that for those who have been through the process before, their perception of the process gets better. All items relating to the system were also compared. All items were responded to more positively in 2008/2009. On the item “The DPAS system needs improving,” more respondents disagreed/strongly disagreed in 2008/2009.

Comparison Between 20072008 and 2008200920072008 Participating Schools Only

0.0010.0020.0030.0040.0050.0060.00

A B C D F

No Res

ponse

Grade

Perc

ent

Overall, what grade would yougive the evaluation process?20072008Overall, what grade would yougive the evaluation process?20082009

Teachers 2007-2008 and 2008-2009 2007-2008 Participating Schools Only

Item Year % % % % % 2007-08 8.94 65.49 20.86 2.98 1.73The system overall is easy to follow 2008-09 10.92 73.17 11.13 1.06 3.712007-08 9.96 65.73 19.29 3.69 1.33The evaluation process provides adequate evidence of

my teaching 2008-09 11.03 66.38 18.03 1.59 2.972007-08 8.31 63.14 22.98 4.16 1.41The evaluation process provides an accurate picture of

my teaching 2008-09 11.24 64.26 19.41 2.01 3.082007-08 7.14 56.31 27.61 4.31 4.63The DPAS II system provides a better picture of my

teaching versus the DPAS I system 2008-09 9.65 61.61 18.77 1.48 8.482007-08 9.18 69.96 16.08 2.59 2.20The Guide is helpful 2008-09 10.92 72.43 11.13 0.53 4.982007-08 8.39 67.37 18.90 2.67 2.67The Guide is easy to understand 2008-09 10.71 69.35 14.10 0.95 4.88


2007-08 12.47 66.75 14.27 4.24 2.27The evaluation did NOT interfere with my duties 2008-09 15.16 66.91 11.88 2.65 3.392007-08 9.88 69.41 15.14 3.76 1.80I perceive the system to be fair and equitable 2008-09 11.13 70.73 11.66 2.12 4.352007-08 11.92 43.76 40.47 0.94 2.90The DPAS evaluation system needs improving 2008-09 6.89 39.24 47.51 1.70 4.672007-08 4.00 62.51 27.29 3.29 2.90I believe the DPAS evaluation system works as intended2008-09 11.56 71.69 11.13 1.91 3.712007-08 3.45 53.02 35.37 4.55 3.61I believe the current DPAS evaluation system should be

continued in its current form 2008-09 4.98 60.02 26.94 2.86 5.20

Teachers General System Items

Strongly Agree Agree Disagree Strongly

Disagree Total

I believe the DPAS evaluation system is being implemented appropriately in my work location. 11.52% 71.69% 13.14% 3.64% 3211 The DPAS evaluation system needs improving. 11.16% 45.32% 42.22% 1.30% 3162 I believe the current DPAS evaluation system should be continued in its current form. 5.39% 56.50% 32.17% 5.93% 3152

Teachers - Overall, what grade would you give the evaluation process? The majority of teachers responded “Agree” or “Strongly Agree” to the item “The DPAS evaluation system needs improving.” However, when asked whether the system works as intended, the majority agreed. Additionally, the majority agreed that it should be


continued in its current form. Similar to 2007-2008, the highest percent of teacher respondents gave the evaluation process a letter grade of “B.”

Specialists General System Items



The DPAS evaluation system needs improving. 12.58% 50.99% 35.10% 1.32% 302

I believe the DPAS evaluation system is being implemented appropriately in my work location. 9.63% 72.43% 13.95% 3.99% 301

I believe the current DPAS evaluation system should be continued in its current form. 3.02% 54.36% 33.89% 8.72% 298

Specialists - Overall, what grade would you give the evaluation process? As with teachers, the majority of specialists believe the DPAS evaluation system needs improving. Fifty-seven percent of specialists responded on the “Agree/Strongly Agree” end of the scale on whether the evaluation system should continue in its current form. Among specialists, 47% gave the evaluation process a grade of “B” and 28% gave the process a grade of “C.”


Administrators General System Items



The DPAS evaluation system needs improving. 14.29% 45.50% 37.57% 2.12% 189

I believe the DPAS evaluation system is being implemented appropriately in my work location. 12.57% 73.30% 9.42% 4.19% 191

I believe the current DPAS evaluation system should be continued in its current form. 7.94% 51.85% 32.28% 7.41% 189

Administrators - Overall, what grade would you give the evaluation process?

The majority of administrators believe that the evaluation system needs improving; however, the majority also believed that the system works as intended and that the system should be continued in its current form. Fifty-one percent of administrators gave the evaluation process a grade of “B.” Similar to 2007-2008 results, the 2008-2009 results in this section of items, across teachers, specialists, and administrators, indicate that there is room for improvement, but that the overall system is good. During some interviews, the interviewers asked teachers, administrators, and specialists what it would take to make the person give a better grade on the process next year. Below are some statements from the interviewees organized into 5 categories: clarity, time consuming/needs simplification, changes and/or a different focus, timeframe of the process, goal setting. The statements presented are just a sampling.


Clarity

Provide calendar with checklist of what is needed and due dates. Additional support and mentoring needs to be provided on the criteria used for DPASII. Need overall calendar of what is required and when. Clarify questions so that you do not need to refer to the ancillary materials. Remove areas that require redundant information. Streamline process to require less paperwork.

Time Consuming/Needs Simplification

Streamline the process for administrators so that it is less time consuming. Process needs to be simplified. Process should require less paperwork. Eliminate duplicity in tracking professional growth through DEEDS and DPASII. Professional requirements for DEEDS is redundant to DPASII. Process seems over complicated for what should be a simple process. Process is time consuming especially for administrators that have to evaluate all of their teachers in a school year. Surveys are time consuming and burdensome. Would like to see paperwork streamlined. Number of meetings can be quite time consuming. Need additional clarification on the number of meetings that are required. Continue to streamline process and forms to require less data entry. Difficult to schedule observations with busy administrators. Process needs to be further simplified so that it is less time consuming.

Changes/Different Focus

Would like more evaluations by different administrators for first year teachers to make sure their teaching is on target. Evaluation process needs to be targeted to the type of teacher. Current process does not provide a way to hold teachers accountable for professional responsibilities (e.g. teachers that frequently show up late.). Need criteria under professional responsibilities that addresses whether or not a teachers abides by district and building policies and procedures. Specialists should be evaluated by other specialists Eliminate Component 5. Provide templates / expectations for lesson plans. Provide guidelines on the amount of information that is required for responses.


Consider using a rubric to evaluate performance vs. satisfactory / unsatisfactory.

Expectations concerning Professional Responsibilities and Performance Data components need to be clarified. Need to clarify the wording of the questions to make them easier to understand what is being required. Specialists are being evaluated on things that they have no control over. It's hard to fit what I do into the five components. Goals (components?) are still geared toward teachers. The process needs to be clearer for specialists and specialists need a better orientation to the process. Principals should not do evaluations. They are not close enough to what teachers are doing. Dept heads or assistant principals should evaluate.

Timeframe

January deadline for Professional Responsibilities component should be moved to March. Paperwork required is time consuming. Evaluation timeline does not take into account the availability of assessment results. Reduce timeframe for providing feedback to teachers. Need to get modifications and materials before school starts. Can't make appropriate adjustments if the information doesn't come until September or October

Goal Setting

Provide specialist teachers with a better idea of what the evaluation process is looking for. Provide additional guidance on goal setting process for specialist where one goal does not apply to all students. Would like to see more feedback through out the year and constructive feedback on how one can improve their teaching. Need more information and guidance on how goals fit into the overall evaluation process. Goals should match school's strategic plan. DPSS II is similar to what we are already doing, but goal setting was new and helpful. Need more clarity in component 5 DPAS II should give examples of goal setting. Was not clear on how the goal should be indicated. More visuals needed. Smaller goals should be required rather than one goal for entire class. Our goals were to be based on Math and English. I find that hard to relate English rubric to Science. If I could set my goals that are


Items were added in 2007-2008 to determine whether the actual time between evaluation components matched staff recommendations as to what should happen. The first item asked the respondents to select an interval of days that reflected the actual number of days between activities. The second item asked the respondents to recommend an interval of days.

Teachers Actual Time Intervals

1-5 days 6-10 days 11-20 days

21-30 days

more than 30 days Total

Scheduling the observation and the pre-observation conference 70.47% 19.53% 5.43% 1.71% 2.86% 3149 Pre-observation conference and the observation 85.00% 10.71% 2.19% 0.45% 1.65% 3146 Observation and the post-observation conference 70.05% 18.51% 5.88% 1.57% 3.98% 3112 Post-observation conference and receipt of the formative feedback form 64.96% 20.35% 7.58% 2.21% 4.90% 3125 Summative conference and receipt of the summative feedback form 59.15% 20.80% 7.23% 2.76% 10.06% 2933

Teachers Staff Recommendation for Intervals

1-5 days

6-10 days

11-20 days

21-30 days

more than 30 days

Don't Know/Don't Care Total

Scheduling the observation and the pre-observation conference 64.37% 20.67% 4.29% 1.72% 1.07% 4.29% 3261 Pre-observation conference and the observation 80.13% 11.04% 1.13% 0.37% 0.25% 3.62% 3261 Observation and the post-observation conference 75.10% 14.01% 2.24% 0.46% 0.46% 3.62% 3261 Post-observation conference and receipt of the formative feedback form 69.52% 18.00% 3.28% 0.80% 0.55% 3.89% 3261 Summative conference and receipt of the summative feedback form 59.89% 20.15% 4.05% 1.75% 2.33% 6.96% 3261 There were not big discrepancies between the actual interval of days between activities and the recommended interval of days. For the most part, the discrepancies were within 6 or less percentage points, and mainly occurred in the first category (1-5 days).


Specialists Actual Time Intervals

1-5 days 6-10 days

11-20 days

21-30 days


Scheduling the observation and the pre-observation conference 57.34% 30.42% 6.29% 1.40% 4.55% 286 Pre-observation conference and the observation 68.77% 18.60% 5.26% 1.75% 5.61% 285 Observation and the post-observation conference 64.39% 21.22% 6.83% 0.72% 6.83% 278 Post-observation conference and receipt of the formative feedback form 63.35% 22.42% 5.69% 1.42% 7.12% 281 Summative conference and receipt of the summative feedback form 53.31% 26.84% 7.72% 2.21% 9.93% 272

Specialists Staff Recommendation for Intervals

1-5 days

6-10 days

11-20 days

21-30 days

more than 30 days

Don't Know/Don't Care Total

Scheduling the observation and the pre-observation conference 53.68% 30.18% 6.32% 1.05% 1.40% 7.37% 285 Pre-observation conference and the observation 70.07% 17.25% 3.52% 0.35% 1.76% 7.04% 284 Observation and the post-observation conference 68.23% 17.33% 4.33% 1.44% 1.44% 7.22% 277 Post-observation conference and receipt of the formative feedback form 63.96% 20.14% 5.65% 1.06% 1.77% 7.42% 283 Summative conference and receipt of the summative feedback form 55.32% 23.76% 7.09% 2.13% 3.90% 7.80% 282 Among specialists, there were minimal differences between the perceived actual interval of days versus the recommended interval of days.

Administrators Select the interval of WORK days that represents the actual time between each pair of activities.

1-5 days

6-10 days

11-20 days

21-30 days


Scheduling the observation and pre-observation conference 71.58% 22.95% 3.83% 1.64% 0% 183 Pre-observation conference and the observation 92.35% 6.01% 1.64% 0% 0% 183

Observation and the post-observation conference 73.33% 22.78% 2.78% 1.11% 0% 180 Post-observation conference and receipt of the formative feedback form 51.37% 38.25% 9.29% 1.09% 0% 183 Summative conference and receipt of the summative feedback form 48.84% 37.79% 9.30% 1.74% 2.33% 172 The majority of administrators selected 1-5 days as the interval that represents the actual time between pairs of activities. This closely aligns with the staff recommendations of the intervals between pairs of activities.


Administrators

Select the interval of WORK days that represent your recommended time between each pair of activities. Staff Recommendation

1-5 days 6-10 days

11-20 days

21-30 days

more than 30 days

Don't Know/ Don't Care Total

(a)

Scheduling the observation and the pre-observation conference 71.04% 20.77% 3.83% 0% 0% 4.37% 183

(b) Preobservation conference and the observation 88.52% 9.29% 0.55% 0% 0.55% 1.09% 183

(c) Observation and the post-observation conference 76.92% 18.13% 2.20% 1.10% 0.55% 1.10% 182

(d)

Post-observation conference and receipt of the formative feedback form 51.10% 33.52% 9.89% 2.75% 1.10% 1.65% 182

(e)

Summative conference and receipt of the summative feedback form 49.72% 31.84% 8.94% 4.47% 1.68% 3.35% 179

Results – Q22 and Q23 22) Does the system enable evaluators to make valid judgments about the

performance of educators? 23) Does the system help evaluators improve the skills and knowledge of those they

evaluate? At the end of the administrator survey, respondents were asked if they were responsible for evaluating other administrators, teachers, and/or specialists. If they answered “yes,” they were branched to a series of items. If they answered “no,” that section of the survey ended. Overall, the evaluator responses were overwhelmingly positive. The following tables reveal the responses to the evaluation items. In 2008-2009, 54 respondents evaluated administrators. Of the 5 major components (as defined in the DPAS II Guide) used in administrator evaluations, which

do you believe are good indicators of performance?

Component 1 - Vision and Goals

Component 2 - Culture of Learning

Component 3 - Management


Responsibilities


Improvement Total

Responses Received in % 69.81% 86.79% 86.79% 67.92% 71.70% 53 Note: Multiple answers per participant possible. Percentages added may exceed 100 since a participant may select moreth an one answer for this question.


Criteria for Evaluating Administrators Please indicate your level of agreement with each of the following statements.



Weighted Score

The written feedback I provide to administrators is aligned with the five components. 25.93% 66.67% 3.70% 1.85% 54 3.19 The oral feedback I provide to administrators is aligned with the five components. 23.08% 65.38% 7.69% 1.92% 52 3.12 I can accurately evaluate administrators using the criteria for the Management component. 22.64% 64.15% 9.43% 1.89% 53 3.1 I can accurately evaluate administrators using the criteria for the Vision and Goals component. 18.52% 70.37% 7.41% 1.85% 54 3.08 I can accurately evaluate administrators using the criteria for the Professional Responsibilities component. 18.87% 67.92% 5.66% 5.66% 53 3.02 I can accurately evaluate administrators using the criteria for the Culture of Learning component. 16.98% 66.04% 13.21% 1.89% 53 3 I can accurately evaluate administrators using the criteria for the Student Improvement component. 17.65% 58.82% 15.69% 5.88% 51 2.9 The “Management” and the “Culture of Learning” components were selected as the best indicator of performance among evaluators of administrators. The least selected component was “Professional Responsibilities.” The majority of administrator evaluators responded that they could accurately evaluate administrators for all criteria in the DPAS II evaluation process. The criterion that had the lowest weighted score was “I can accurately evaluate administrators using the criteria for the Student Improvement component.” Additionally, all of the evaluators responded on the “Agree/Strongly Agree” end of the scale for alignment of written and oral feedback with the five components.

Evaluators Administrators (System, Documentation, Data, Feedback):



Weighted Score

Administrators are accepting of their evaluation feedback. 29.63% 61.11% 7.41% 0% 54 3.23 The timing of administrator conferences is good. 20.37% 64.81% 9.26% 3.70% 54 3.04 Administrators are able to provide the evidence and documentation I need to evaluation them accurately. 14.81% 68.52% 14.81% 0% 54 3 The evaluation process provides adequate evidence of administrators' performance. 13.21% 73.58% 9.43% 1.89% 53 3 The administrator forms are easy to complete. 14.81% 66.67% 14.81% 1.85% 54 2.96 The evaluation process provides an accurate picture of administrators' performance. 14.81% 64.81% 14.81% 3.70% 54 2.92 There are adequate resources for administrators to implement improvement plans. 9.80% 68.63% 17.65% 1.96% 51 2.88 Administrators are able to complete the data documentation requirements without difficulty. 15.09% 58.49% 22.64% 1.89% 53 2.88 Evaluators were asked to respond to a series of items that dealt with the system, documentation, data, and feedback mechanisms. Based on the responses, it appears that administrators are accepting of their evaluation feedback. Conversely, there was


less support for items relating to adequate resources to implement improvement plans and completing the data documentation without difficulty. There were 181 respondents who evaluated teachers. When asked which criteria are good indicators of performance for teachers, 96% selected “Instruction.” Only 54% selected “Professional Responsibilities.”

Evaluating Teachers Of the 5 major components (as defined in the DPAS II Guide) used in teacher evaluations,

which do you believe are good indicators of performance?


Classroom Environment Instruction

Professional Responsibilities

Student Improvement Total

Responses Received in % 86.74% 85.08% 96.13% 54.14% 70.17% 181

Evaluating Teachers



Weighted Score

The written feedback I provide to teachers is aligned with the five components. 41.99% 55.80% 1.10% 0.55% 181 3.4 The oral feedback I provide to teachers is aligned with the five components. 38.12% 60.22% 0.55% 0.55% 181 3.37 I can accurately evaluate teachers using the criteria for the planning and preparation component. 33.52% 63.19% 2.20% 0.55% 182 3.3 I can accurately evaluate teachers using the criteria for the classroom environment component. 33.15% 63.54% 2.21% 0.55% 181 3.3 I can accurately evaluate teachers using the criteria for the instruction component. 31.64% 64.97% 2.26% 0.56% 177 3.28 I can accurately evaluate teachers using the criteria for the professional responsibilities component. 22.53% 67.03% 8.24% 1.65% 182 3.11 I can accurately evaluate teachers using the criteria for the student improvement component. 23.08% 57.14% 17.58% 1.65% 182 3.02

As with the teachers’ responses regarding criteria that are good indicators of performance, the professional responsibilities and the student improvement components received the least support among teacher evaluators. The large majority of teacher evaluators responded on the “Agree/Strongly Agree” end of the scale on being able to use the criteria to accurately evaluate the components. Additionally, the respondents answered positively on providing written and oral feedback that is aligned with the 5 components.


Evaluators Teachers (System, Documentation, Data, Feedback):



Weighted Score

Teachers are accepting of their evaluation feedback. 23.20% 71.27% 4.42% 0.55% 181 3.18 Teachers are able to provide the evidence and documentation I need to evaluate them accurately. 21.23% 70.39% 6.70% 1.12% 179 3.12 The teacher forms are easy to complete. 18.23% 63.54% 17.13% 0.55% 181 3 The evaluation process provides adequate evidence of teachers' performance. 16.67% 66.11% 15.56% 1.11% 180 2.99 There are adequate resources for teachers to implement improvement plans. 15.25% 70.06% 12.43% 1.69% 177 2.99 The evaluation process provides an accurate picture of teachers' performance. 15.56% 63.33% 19.44% 1.11% 180 2.94 The timing of teacher conferences is good. 17.88% 60.89% 16.20% 4.47% 179 2.93 Teachers are able to complete the data documentation requirements without difficulty. 14.36% 55.80% 25.41% 3.87% 181 2.81 There is congruence with the results of school level data and classroom data. 6.21% 67.80% 20.34% 5.08% 177 2.76 Classroom level DSTP data provides an accurate picture of student progress. 5.08% 34.46% 46.89% 12.99% 177 2.32 Among the teacher evaluators, there were positive responses relating to the system, documentation, data, and feedback mechanisms. The highest mean score was on the item “Teachers are accepting of their evaluation feedback.” The next highest mean scores were on the items: 1) “Teachers are able to provide the evidence and documentation I need to evaluate them accurately,” and 2) “The teacher forms are easy to complete.” The lowest mean scores were for the data related items. There were 152 respondents who evaluate specialists.

Evaluating Specialists Of the 5 major components (as defined in the DPAS II Guide) used in specialist evaluations,

which do you believe are good indicators of performance?


Professional Practice and Delivery of

Service

Professional Collaboration and

Consultation Professional


Improvement Total Responses Received in % 75.00% 92.76% 86.18% 73.03% 55.92% 152 Among specialist evaluators, the “Student Improvement” component was the least selected component for being a good indicator of performance. The component most selected was “Professional Practice and Delivery of Service.” Evaluators of specialists responded positively to the items relating to the evaluation criteria. The item with the most desirable responses was “The written feedback I provide to specialists is aligned with the five components.”


Evaluating Specialists



Weighted Score

The written feedback I provide to specialists is aligned with the five components. 28.95% 67.11% 2.63% 1.32% 152 3.24 The oral feedback I provide to specialists is aligned with the five components. 28.86% 67.11% 2.68% 1.34% 149 3.23 I can accurately evaluate specialists using the delivery of service component. 21.85% 68.87% 7.95% 1.32% 151 3.11

I can accurately evaluate specialists using the criteria for the professional collaboration and consultation component. 21.71% 68.42% 8.55% 1.32% 152 3.11

I can accurately evaluate specialists using the criteria for the planning and preparation component. 22.37% 66.45% 9.87% 1.32% 152 3.1

I can accurately evaluate specialists using the criteria for the professional responsibilities component. 21.71% 68.42% 7.89% 1.97% 152 3.1

I can accurately evaluate specialists using the criteria for the student improvement component. 17.22% 44.37% 31.13% 7.28% 151 2.72

Evaluators Specialists (System, Documentation, Data, Feedback):



Weighted Score

Specialists are accepting of their evaluation feedback. 19.08% 76.88% 3.47% 0.58% 173 3.14

Specialists are able to provide the evidence of documentation I need to evaluate them accurately. 15.82% 72.88% 10.17% 1.13% 177 3.03 The timing of specialists conferences is good. 17.24% 66.67% 13.79% 2.30% 174 2.99 The evaluation process provides adequate evidence of specialists' performance. 17.34% 67.05% 12.72% 2.89% 173 2.99 The specialist forms are easy to complete. 16.48% 66.48% 15.34% 1.70% 176 2.98 There are adequate resources for specialists to implement improvement plans. 12.87% 70.18% 13.45% 3.51% 171 2.92 The evaluation process provides an accurate picture of specialists' performance. 16.48% 60.23% 20.45% 2.84% 176 2.9 Specialists are able to complete the data documentation requirements without difficulty. 12.50% 64.20% 19.89% 3.41% 176 2.86 Similar to the responses from evaluators of teachers and administrators, the evaluators of specialists responded positively to the item “Specialists are accepting of their evaluation feedback.” The item that received the least support was “Specialists are able to complete the data documentation requirements without difficulty.”


Evaluators Documentation- General: Please indicate your level of agreement with each of the following statements.

Please indicate your level of agreement with each of the following statements.



Weighted Score

I have access to the information I need to complete the forms. 16.32% 77.37% 4.21% 2.11% 190 3.08

The forms play an important role in the overall evaluation. 19.27% 68.23% 10.94% 1.56% 192 3.05

The information on the forms is consistent with determining the outcome of the evaluation. 13.23% 74.60% 10.05% 2.12% 189 2.99

The required paperwork is relevant to the evaluation. 12.63% 65.79% 17.37% 4.21% 190 2.87

The forms make the process easy to implement. 14.74% 60.00% 22.11% 3.16% 190 2.86

I am able to complete paperwork in a reasonable time period. 8.38% 63.87% 18.32% 9.42% 191 2.71

The workload is manageable. 6.38% 52.13% 31.91% 9.57% 188 2.55

The time it takes to complete the DPAS II paperwork requirements is reasonable. 6.25% 48.96% 31.25% 13.54% 192 2.48

Evaluators

Feedback- General: Please indicate your level of agreement with each of the following statements.

Please indicate your level of agreement with each of the following statements



Weighted Score

The number of conferences/conversations is adequate. 10.93% 73.22% 13.66% 2.19% 183 2.93

Overall, Improvement Plan recommendations are perceived to be useful. 9.71% 70.29% 15.43% 4.57% 175 2.85

EvaluatorsSystem Related Items- General:

Please indicate your level of agreement with each of the following statements.



Weighted Score

The Guide is helpful. 24.47% 71.81% 3.72% 0% 188 3.21 The Guide is easy to understand. 22.87% 71.28% 5.85% 0% 188 3.17 The appeals process is fair. 14.37% 81.03% 4.02% 0.57% 174 3.09 The training for the districts was timely. 14.29% 76.72% 8.47% 0.53% 189 3.05 Training in the process is adequate. 15.51% 74.33% 10.16% 0% 187 3.05

The time required in the appeals process is reasonable. 10.92% 82.18% 5.75% 1.15% 174 3.03

The system is easy to follow. 11.89% 75.14% 11.89% 1.08% 185 2.98

The system is fair and equitable among teachers, administrators, and specialists. 11.48% 73.77% 10.38% 4.37% 183 2.92

The DPAS II system is more appropriate than the DPAS I system. 14.29% 63.74% 19.78% 2.20% 182 2.9



By far, the lowest weighted score in the general items was on “The time it takes to complete the DPAS II paperwork is reasonable,” followed by “The workload is manageable.” However, with the exception of the data related items, a large majority responded on the agree end of the scale on most all items for teachers, administrators, and specialists.

APPENDIX A

Pilot, Phase I, Other Statewide Comparison Pilot Districts: Appoquinimink and Caesar Rodney Phase I: Colonial, Lake Forest, Laurel, Sussex Tech, MOT Charter, Providence Creed Academy Charter, Sussex Academy of Arts and Science plus Charters Other – All other districts

Strongly Agree to Strongly Disagree Items District Grouping Means Pilot Phase I Other The five components used to evaluate my performance are understandable. 3.19 3.25 3.15 The five components used to evaluate my performance are reasonable indicators of my effectiveness. 2.99 3.05 2.99 The criteria used to evaluate me for the planning and preparation component can be accurately judged by my evaluator. 3.07 3.02 2.94 The criteria used to evaluate me for the classroom environment component can be accurately judged by my evaluator. 3.12 3.15 3.05 The criteria used to evaluate me for the instruction component can be accurately judged by my evaluator. 3.17 3.18 3.08 The criteria used to evaluate me for the professional responsibilities component can be accurately judged by my evaluator. 2.98 3.02 2.91 The criteria used to evaluate me for the student improvement component can be accurately judged by my evaluator. 2.69 2.75 2.66 Applying all five components in my work is easy. 2.79 2.81 2.74 The written feedback I receive is aligned with the five components. 3.20 3.21 3.08 The oral feedback I receive is aligned with the five components. 3.18 3.17 3.06 The forms play an important role in the overall evaluation. 2.88 2.87 2.74 I am able to provide the evidence and documentation needed by my evaluator for him/her to accurately determine my effectiveness. 3.09 3.16 3.05 I am able to provide evidence of my practice through artifact. 3.06 3.13 3.01 The time it takes to complete the DPAS II paperwork requirements is reasonable. 2.74 2.86 2.57 The forms are easy to complete. 2.86 2.92 2.73 I have access to the information I need to complete the forms. 3.06 3.15 3.01 The forms make the process easy to implement. 2.84 2.90 2.69 The information on the forms is consistent with determining the outcome of the evaluation. 2.92 3.00 2.85 The required paperwork is relevant to the evaluation. 2.89 2.94 2.76 My evaluator completes paperwork in a reasonable time period. 3.13 3.23 3.07 My evaluator handles the workload effectively. 3.12 3.19 3.04 Overall, the feedback I receive is adequate. 3.18 3.17 3.09


The oral feedback I receive is useful and applicable. 3.18 3.18 3.08 The written feedback I receive is useful and applicable. 3.14 3.15 3.05 In general, the conferences are valuable. 3.16 3.13 3.03

Strongly Agree to Strongly Disagree Items District Grouping Means Pilot Phase I Other The forms completed after conferences are valuable. 2.99 2.96 2.83 I am able to provide evidence of my practice through discussion. 3.25 3.23 3.15 The timing of the conferences is good. 3.08 3.10 3.00 The number of conferences/conversations with my evaluator is adequate. 3.10 3.09 3.01 The system overall is easy to follow. 2.95 3.00 2.77 The evaluation process (observations, documentation, and conferences) provides adequate evidence of my teaching. 2.87 2.92 2.81 The evaluation process (observations, documentation, and conferences) provides an accurate picture of my teaching. 2.87 2.88 2.76 The DPAS II system provides a better picture of my teaching versus the DPAS I system. 2.92 2.84 2.64 The Guide is helpful. 2.97 3.01 2.92 The Guide is easy to understand. 2.93 2.96 2.84 The evaluation did NOT interfere with my duties. 2.92 3.03 2.88 I perceive the system to be fair and equitable. 2.92 2.99 2.87 The DPAS evaluation system needs improving.** 2.59 2.53 2.72 I believe the DPAS evaluation system is being implemented appropriately in my work location. 2.95 3.00 2.88 I believe the current DPAS evaluation system should be continued in its current form. 2.70 2.72 2.57 The training was timely. 2.88 2.90 2.82 Training in the process is adequate. 2.83 2.86 2.74 Additional training would make me feel more competent in the process. 2.41 2.52 2.54 Classroom level DSTP data gives me an accurate picture of my students progress. 2.21 2.24 2.22 I was able to complete the data documentation requirements without difficulty. 2.64 2.74 2.62 There was enough training and/or support for me to accurately complete the forms related to student improvement. 2.63 2.73 2.60 DSTP data helps me adjust instruction for my students. 2.34 2.44 2.35 There was congruence with the results of school level data and my classroom data. 2.58 2.68 2.59 The Improvement Plan process helped direct my professional growth goals. 1.83 2.71 2.34 The Improvement Plan recommendations were useful. 2.17 2.71 2.38 There are adequate resources to implement improvement plans. 2.00 2.57 2.26 The Improvement Plan outlined measurable goals for me to work toward achieving. 2.33 3.00 2.56 The challenge process was fair. 2.71 3.00 2.87 The time required in the challenge process was reasonable. 2.57 3.00 2.88 The outcome of my challenge was judged accurately. 2.86 3.00 2.89



The online manual was useful. 2.98 3.00 2.94 The online manual was easy to use. 2.98 2.99 2.92 The short videos were helpful. 2.87 2.88 2.81 The training materials were helpful. 2.95 2.99 2.94 The FAQs addressed my questions. 2.94 2.96 2.90

Strongly Agree to Strongly Disagree Items District Grouping Means Pilot Phase I Other The website provides me with all the information I need on DPAS II. 2.94 2.98 2.88 *Note: Items coded as Strongly Agree=4, Agree=3, Disagree=2, Strongly Disagree =1 **Lower mean is desirable

Letter Grade Items District Grouping Means Pilot Phase I Other

Overall, what grade would you give the evaluation process? 3.82 3.86 3.60 *Note: Letter grades were coded as A=5, B=4, C=3, D=2, F=1

delaware performance appraisal system second edition (dpas ii) · 2014. 8. 28. · 4. separate the...

Documents