universidad del turabo school of education educational ......of program course content with a...

Universidad del Turabo

School of Education

Educational Leadership Program

Council for the Accreditation of Educator Preparation

CAEP/TEAC Annual Report 2018

Submitted by

Dr. Rafael Cartagena-Rodríguez

SOED CAEP/TEAC Accreditation Committee Chair

Prof. Israel Rodríguez-Rivera

SOED Dean

TABLE OF CONTENTS

Page

Introduction

SOED Enrollment and Graduation Data 1

Assessment Results

Claim 1 1

Claim 2 5

Claim 3 5

Claim 4 7

Claim 5 8

Inventory of evidence 10

UT SOED ELP CAEP/TEAC Annual Report 2016-2017 (Submitted 2018) 1

Council for the Accreditation of Educator Preparation

CAEP/TEAC Annual Report 2015

Educational Leadership Program

Introduction

The School of Education (SOED) Educational Leadership Program (ELP) is comprised of two degrees: Master in Education (MEd) with specialization in Educational Administration and Doctorate in Educational

Leadership (EdD). The SOED Educational Leadership Program did not suffer changes during the report period:

academic year 2015-16. This annual report will include data analysis for both degrees.

SOED Enrollment and Graduation Data

The number of enrolled students in SOED Educational Leadership Program increased from a combined

number of 129 students in both degrees to 133 students in the last academic year. The MEd program enrollment

increased from 55 students (2015-16) to 64 (2016-17). The changes in regulations regarding the requirements for

becoming a school director, thus, the number of students seeking for a School Director Certification and the

availability of those positions in the Puerto Rico Department of Education (PRDE) still the main reasons for enrollment

changes between academic years. Also, mobility and demographic changes due economic recession in Puerto Rico

has had an impact on education programs. EdD program had a small decrease from 74 students (2015-16) to 69 (2016-16).

Table 1: Enrollment in the ELP for Academic Years: 2014-2015, 2015-16 and 2016-17

Degree 2014-15 2015-16 2016-17

MEd Educational Administration 63 55 64

Ed D Educational Leadership 69 74 69

Total 132 129 133

The number of students who completed the SOED Educational Leadership Program increased by 19 students from

2015-16 to 2016-17. All 19 students were enrolled in the MEd degree. The number of graduates from the Ed.D.

program was equal to the previous year: 11 students.

Table 2: Graduation Data of ELP Students for Academic Years: 2014-2015, 2015-16 and 2016-17

Degree 2014-15 2015-16 2016-17

MEd Educational Administration 7 5 24

Ed D Educational Leadership 9 11 11

Total 16 16 35

Assessment Results

Claim 1 Professional Knowledge: The faculty claims that the graduates of the MA degree in Educational Administration and the graduates of the Ed. D. in Educational Leadership understand the political, social, economic

and cultural complexity of educational enterprises, and can use that understanding to solve problems and create a

school environment that enhances learning.

Grade Distributions

Students in the master and doctoral level must approve their courses with a minimum grade of C and must

have a GPA of 3.00 as stated in the academic norm. The ABC rate in the grade distributions in the MEd professional

courses achieved the expected 80% in all the professional courses.


Table 3: Grade Distribution in the MEd Professional Courses: 2016-17

Course A % B % C % ABC % D % F % Total

Educ 503 38 100.00 38 100.0 38

Educ 504 31 79.49 7 17.95 38 97.44 1 2.56 39

Educ 506 13 76.47 3 17.65 16 94.12 1 5.88 17

Educ 510 40 90.91 3 6.82 43 97.73 1 2.27 44

Educ 519 37 88.10 5 11.90 42 100.00 42

Educ 520 33 94.29 1 2.86 34 97.14 1 2.86 35

Educ 702 22 57.89 16 42.11 38 100.0 38

Educ 705 11 91.67 1 8.33 12 100.0 12

Practicum 34 100.0 34 100.0 34

The percentage of students enrolled in the MEd professional courses during the academic year 2016-17 that comply with the expected minimum GPA of 3.00 is 100% (81/81 students).

Table 4: Descriptive Statistics for GPA in Professional Courses of the MEd Degree

Statistics Value

No students 81

Mean 3.89

Median 4

Mode 4

Standard Deviation 0.18

Variance 0.03

Range 1

Maximum 4

Minimum 3

Number of Students GPA > 3.00 81

% Students GPA > 3.00 100

* Median is 4.00 because of the large amount of students with a GPA of 4.00 (50/81 students)

The ABC-Grade rate in the grade distributions in the EdD professional courses achieved the expected 80%

in all the professional courses.

Table 5: Grade Distribution in the EdD Professional Courses: 2016-17

Subject A % B % C % ABC % D % F % Total

Educ 801 7 100.00 7 100.00 7

Educ 802 6 40.00 6 40.00 2 13.33 14 93.33 1 6.67 15

Educ 803 6 85.71 1 14.29 7 100.00 7

Educ 804

Educ 805 7 87.50 1 12.50 8 100.00 8

Educ 806 7 77.78 2 22.22 9 100.00 9

Educ 807 9 100.00 9 100.00 9

Educ 804 was not offered during the academic year 2016-17.


Table 6: Descriptive Statistics for GPA in Professional Courses of the EdD Degree

Statistics Value

No students 30

Mean 3.59

Median 3.75

Mode 4

Standard Deviation 0.58

Variance 0.33

Range 2

Maximum 4

Minimum 2

Number of Students GPA > 3.00 25

% Students GPA > 3.00 83.33

The percentage of students enrolled in the EdD professional courses during the academic year 2016-17 that comply

with the expected minimum GPA of 3.00 is 83.33% (25/30 students).

MEd Comprehensive examinations/Professional Portfolio

Students at the MEd have the choice to take the comprehensive exam or approve the course Educ 604:

Knowledge Integration Seminar. In the MEd program, no students have taken the comprehensive exam during the past three academic years. Instead, they completed the course Educ 604. The approval of course Educ 604 requires

the design of an electronic portfolio that compiles previous tasks from their courses and practicum, and the completion

of the following tasks: Reflection Paper (using the Curere strategy), Service Learning Plan, Specialized Topic Paper,

and a Professional Dispositions Paper. Rubrics for each task were developed and are used as a homogeneous method

to evaluate them.

In the MEd program, no students took the comprehensive exam during the past three academic years. Instead,

they completed the course Educ 604. In course Educ 604; 18 out of 18 students (100%) completed the portfolio

requirements of course Educ 604 and approved the course.

EdD Comprehensive examinations/Professional Portfolio

The comprehensive exam includes a Social Context and a Leadership Component. Students are presented with two questions in each component and must select one to answer. Experts of each area carefully construct the

questions; experts who have taught the courses to the students that will be examined. Then, they revised the questions

in a teamwork session under the direction of the Doctoral Program Coordinator. Validity of the questions relies on

the expert’s judgment and ability to align the main topics in the courses with the questions. Final approval of the test

requires the approval of both components.

Table 7: Descriptive Statistics for the EdD Comprehensive Exam: 2014-2015 and 2015-16

Exam Social Context Leadership

Statistics 2015-16 2016-17 2015-16

N 9 7 9 7

Mean 84.67 81.86 83.44 82.00

Median 85.00 80.00 84.00 81.00

Mode 85 80 78b 80

Std. Deviation 5.454 4.981 3.504 2.517

Variance 29.75 24.810 12.28 6.333

Range 18 14 10 6

Minimum 74 74 78 80

Maximum 92 88 88 86

In 2016-2017, 100% (7/7 students) approved the exam. Results from 2015-16 and 2016-17 comprehensive

exam components were compared. For the Social Context component, there was no significant variability (F= 0.011,

p= 0.920) and no statistical significant differences between scores in the t-test for independent samples (t=1.061,


df=14, Sig 2-tailed =0.367). For the Leadership component, there was no significant variability (F=0.745, p= 0.403)

and no significant difference in the t-test for independent samples (t=0.919, df=14, Sig 2-tailed =0.374).

Table 8: Results of the Independent Samples t-Test for the Comprehensive Exam’s Social Context and

Leadership Components (2015-16; 2016-17

Independent Samples Test

Levene's Test for

Equality of Variances t-test for Equality of Means

F Sig. t df Sig. (2-tailed) Mean Difference

SContext Equal variances

assumed

.011 .920 1.061 14 .307 2.810

Equal variances

not assumed

1.073 13.563 .302 2.810

Leadership Equal variances

assumed

.745 .403 .919 14 .374 1.444

Equal variances

not assumed

.959 13.949 .354 1.444

Dissertation Courses

MEd students can use EDUC 617 as an integration seminar in substitution of a thesis. The student will

complete a non-interactive research project in educational administration. This course gives the student a synthesis of program course content with a research based focus. In the academic year 2016-17, 20 out of 21 students (95.24%)

completed the research project and approved the Educ 617 seminar. The student got an IP grade (In Progress) in the

course and will receive additional support to complete the research project.

Seven (7) out of nine (9) doctoral students immersed in the final phase of their thesis dissertation approved

satisfactorily their thesis defense and became graduates at the academic year 2016-17 (7/9 =77.78%).

Pre and Posttest using ISLLC Standards for Educational Leaders

The ISLLC Standards for Educational Leaders were used as guidelines in the development of twenty (20)

items in the pre and post-test administered to all candidates in the MEd Practicum. Data files were damaged during

Hurricane María, therefore, there is no analysis for the administration of the pre/post ISLLC scores.

MEd Course EDUC 515 - Practicum

Students enrolled in the practicum experience complete the following tasks: Reflective dairies, Pre/Post-Test

based on the ISLLC Standards and the Final Report. The University Supervisor and the Cooperative Director evaluate

the student based on his/her performance during the process through a collaborative discussion. Hundred percent of

the students enrolled in Practicum approved the course: 34/34 students.

Table 9: Grade Distribution in the MEd Practicum – Educ 515

Year A % B % C % ABC % D % F % Total

2014-15 19 100.0 19 100.0 19

2015-16 29 93.50 29 93.50 2 6.50 31

2016-17 34 100.0 34 100.0 34

Total 82 97.62 82 97.62 2 2.38 84


Claim 2 Strategic Decision Making. The faculty claims that the graduates understand educational structures and

processes use this understanding to promote the success of every student, and gather data to measure the

effectiveness of these efforts.







exam components were compared. For the Social Context component, there was no significant variability (F= 0.011,

p= 0.920) and no statistical significant differences between scores in the t-test for independent samples (t=1.061,



Practicum Evaluations Students enrolled in the practicum experience complete the following tasks: Reflective dairies, Pre/Post-Test


the student based on his/her performance during the process through a collaborative discussion. Hundred percent of the students enrolled in Practicum approved the course: 34/34 students.

Data from Table 9: Grade Distribution in the MEd Practicum – Educ 515


2016-17 34 100.0 34 100.0 34

Dissertation Courses MEd students can use EDUC 617 as an integration seminar in substitution of a thesis. The student will

complete a non-interactive research project in educational administration. This course gives the student a synthesis

of program course content with a research based focus. In the academic year 2016-17, 20 out of 21 students (95.24%)



Seven (7) out of nine (9) doctoral students immersed in the final phase of their thesis dissertation approved

satisfactorily their thesis defense and became graduates at the academic year 2016-17 (7/9 =77.78%).

Pre and Posttest using ISLLA Test

The ISLLC Standards for Educational Leaders were used as guidelines in the development of twenty (20)

items in the pre and post-test administered to all candidates in the MEd Practicum. Data files were damaged during

Hurricane María, therefore, there is no analysis for the administration of the pre/post ISLLC scores.

Claim 3: Caring Leadership Skills and Multicultural Perspectives and Accuracy: The faculty claims that the

program prepares educational leaders who will act in an ethical, fair and trustworthy manner to promote success of

every student and are understanding and sensible to parents’, students’ and teachers’ cultural, racial, gender and

individual differences.

Professional Portfolio

Students at the MEd have the choice to take the comprehensive exam or approve the course Educ 604:

Knowledge Integration Seminar. In the MEd program, no students have taken the comprehensive exam during the

past three academic years. Instead, they completed the course Educ 604. The approval of course Educ 604 requires

the design of an electronic portfolio that compiles previous tasks from their courses and practicum, and the completion of the following tasks: Reflection Paper (using the Curere strategy), Service Learning Plan, Specialized Topic Paper,

and a Professional Dispositions Paper. Rubrics for each task were developed and are used as a homogeneous method

to evaluate them. In course Educ 604; 18 out of 18 students (100%) students approved the course in 2016-17.


MEd Practicum







2016-17 34 100.0 34 100.0 34

Grades Distributions

MEd Courses

Educ 504 – Leadership, Communication and Teamwork C grade is the minimum grade required to approve the course. In the past three years, grade distributions of

the course surpassed the expected 80% rate in ABC grades. minimum grades required to approve the course.

Table 15: Grade Distribution in the MEd Course Educ 504


2014-15 16 10.0 16 100.0 16

2015-16 15 75.0 2 10.0 17 85.0 3 15 20

2016-17 31 79.49 7 17.95 38 97.44 1 2.56 39

Total 62 82.67 9 12.00 71 94.67 4 5.33 75

Educ 515 – Practicum



2016-17 34 100.0 34 100.0 34

EdD Courses

C grade is the minimum grade required to approve the course. In the past three years, 14/15 (93.33%) students

enrolled in the course Educ 714 approved with at least the minimum grade required.

Table 16: Grade Distribution in the EdD Course Educ 714


2014-15 10 100.0 10 100.0 10

2015-16 2 66.67 1 33.33 3 100.0 3

2016-17 1 50.00 1 50.00 1 50.00 2

Total 13 86.67 1 6.67 14 93.33 1 6.67 15

C grade is the minimum grade required to approve the course. In the past three years, three students enrolled

in the course Educ 715 approved with at least the minimum grade required.



2014-15 0

2015-16 1 100.0 1 100.0 1

2016-17 1 50.00 1 50.00 2 100.0 2

Total 2 66.67 1 33.33 3 100.0 3


C grade is the minimum grade required to approve the course. In the past three years, 28 students enrolled




2014-15 2 20.00 6 60.00 2 20.00 10 100.0 10

2015-16 5 38.46 7 53.85 1 7.69 13 100.0 13

2016-17 1 20.00 4 80.00 5 100.0 5

Total 8 28.57 17 60.71 3 10.71 28 100.0 28

C grade is the minimum grade required to approve the course. In the past three years, 25 students enrolled




2014-15 12 92.3 1 7.7 13 100.0 13

2015-16 4 100.0 4 100.0 4

2016-17 9 100.0 9 100.0 9

Total 25 100.0 25 100.0 25





2014-15 1 33.33 1 33.33 1 3.33 1 33.33 3

2015-16 2 100.0 2 100 2

2016-17 2 40.00 1 20.00 2 40.00 5 100.0 5

Total 5 55.56 1 11.11 2 22.22 7 77.78 1 11.11 1 11.11 9





2014-15 10 100.0 10 100.0 10

2015-16 12 92.31 12 92.31 1 7.69 13

2016-17 9 100 9 100.0 9

Total 31 96.88 31 96.88 1 3.12 32

Claim 4: Learning How to Learn: The faculty claims that the constructivist learning philosophy

enhances students’ abilities to continue learning after completion of programs.

Work Samples from courses Educ 604 – Knowledge Integration Seminar and Educ 617 – Classroom Research

Seminar and Clinical Experiences Educ 515 – Practicum






MEd Educ 617

MEd students can use EDUC 617 as an integration seminar in substitution of a thesis. The student will

complete a non-interactive research project in educational administration. This course gives the student a synthesis

of program course content with a research based focus. In the academic year 2016-17, 20 out of 21 students (95.24%)



MEd Course EDUC 515 - Practicum







2016-17 34 100.0 34 100.0 34







exam components were compared. For the Social Context component, there was no significant variability (F= 0.011, p= 0.920) and no statistical significant differences between scores in the t-test for independent samples (t=1.061,



Claim 5 Technology: The faculty claims that students use instructional technology throughout

coursework as an integral part of the program.

Grades Distributions in MEd Courses Educ 501: Curriculum Design and Principles



Table 22: Grade Distribution in the MEd Course Educ 501


2014-15 0

2015-16 7 53.85 4 30.77 11 84.64 1 7.69 1 7.69 13

2016-17 20 100.0 20 100.0 20

Total 27 84.38 4 12.50 31 96.88 1 3.12 32


EdD Courses Educ 808: Educational Computing

C grade is the minimum grade required to approve the course. In the past three years, all (5/5) students




2014-15 1 50.0 1 50.0 2 100.0 2

2015-16 0

2016-17 3 100.0 3 100.0 3

Total 4 80.00 1 20.00 5 100.0 5

Educ 809: Instructional Design, C grade is the minimum grade required to approve the course. In the past three years, all (16/16) students




2014-15 8 80.0 2 20.0 10 100.0 10

2015-16 4 100 4 100.0 4

2016-17 2 100.0 2 100.0 2

Total 14 87.50 2 12.50 16 100.0 16

Educ 810: Technology Media in Education and Training

C grade is the minimum grade required to approve the course. In the past three years, all (5/5) students




2014-15 4 100.0 4 100.0 4

2015-16 0

2016-17 1 100.0 1 100.0 1

Total 5 100.0 5 100.0 5

Educ 811: Theory and Practice of Distance Education

C grade is the minimum grade required to approve the course. During 2016-2017, there were no Students

from the program enrolled in Educ 811. 8 approved with at least the minimum grade required.



2014-15 4 80.0 1 20.0 5 100.0 5

2015-16 0

2016-17

Total 4 80.0 1 20.0 5 100.0 5

UT SOED CAEP/TEAC ELP Annual Report 2016-2017 (Submitted 2018) 10

Inventory of evidence

Table 18: General Description of the Evidence to Support the Claims

Evidence Relevance Verifiable Representative Cumulative Actionable

Grade

distributions

With added confidence by

using in combination with

other evidence give clear

picture of student achievement.

Easy to verify electronically

as it is certified by the

Registrar.

All students receive grades in

their courses.

Provide a consistent history of

student achievement over time

Yield information for

curriculum and academic

regulations revision; provide

information on faculty development needs.

Comprehensive

examinations or

EDUC 604

Students’ demonstrate

knowledge and skills.

Scores and test guidelines are

available to review

MA students take

comprehensive examination or

EDUC 604; all doctoral

students are required to take

The Comprehensive

Examination.

Provide a historical record of

mastery of content and skills.

Allows faculty to calibrate

students’ strengths and

weaknesses and take action in

program revision.

Capstone projects

and Professional

Portfolios

Reinforce grades and

comprehensive examination in

understanding students’

acquisition of skills and

knowledge.

Portfolios are easily verifiable

by independent evaluators.

All MA and Doctoral

students’ are required to

complete a Capstone Project

and Portfolio.MA students

start a portfolio in EDUC 504,

doctoral program start in EDUC 716.

Allow students to create a

reflective history of academic

life in the program from

beginning to end.

Allows for students

recommendations for revising

courses and processes.

Dissertation

Courses

Students demonstrate mastery

of the process of acquiring,

processing and creating new

knowledge

The document is published

and presented to the

educational community.

All doctoral students complete

a dissertation.

Students have to accumulate

different skills and knowledge

to write a dissertation.

Dissertations’

recommendations are

actionable. Quality of

document reflects quality of

program and provides new

knowledge and

recommendations for action.

End-of-program

survey

Information is essential for

understanding students’ self-

assessment of knowledge and

quality od program.

Results are presented for

faculty to discuss and take

action.

All students complete the end

of program survey.

Accumulated and discussed

results over the years reinforce

or contradict evidence of

student achievement.

Information is used to revise

and improve programs.

Pre and Posttest

using ISLLC

Standards for Educational

Leaders

Data was damaged during the

hurricane. Analysis was not completed.

MA Internship’

Results

Theory is put into practice.

Educational output is seen in

All evaluations instruments

produced during internship are

All MA students complete the

internship.

The internship scheduled at

the end of MA program

Students’ strengths and

weakness are shown in


real time in a real educational

setting.

kept in SOED and are

available for examination.

provides a comprehensive

view of the candidates

capabilities developed over

time in program.

internship evaluations reports

indicating faculty the direction

of program adjustments.

Work samples

MA Students have to

demonstrate skills by

producing specific products in

selected courses.

Work samples are kept by

faculty members and are

available for examination.

All MA students have to take

EDUC 604, 617 and 515.

All these courses required

acquired knowledge and skills

during program in order to

produce three different work samples.

Evaluation documents of

courses provide information

for program revision.

Table 19: Inventory of evidence

Inventory: Status of Evidence from measures and indicators for TEAC Quality Principle I

Type of Evidence Available and in the Brief Not Available and Not in the Brief

Note: items under each category are

examples. Program may have more or

different evidence

Relied on

Reasons for including the results in the

Brief

Location in Brief

Not Relied on Reasons for not

relying on this

efficiency

Location in Brief

For future use

Reasons for including in

future Briefs

Not for future use

Reasons for not including in future

Briefs

Grades

1. Student grades and grade point average It is a solid evidence of

acquired knowledge

4, 5, 6,

7, 8, 9

Scores on standardized tests

2. Student scores on standardized license or

board examinations

Supports our claim of

acquired professional

knowledge

3. Student scores on undergraduate and/or

graduate admission tests of subject

matter knowledge and aptitude.

4. Standardized scores and gains of the

program graduates’ own pupils.

Very difficult to obtain data

especially when graduates are in

power positions.

Ratings

5. Ratings of portfolios of academic and clinical accomplishments

Students can demonstrate knowledge and professional

skills

4,5

6. Third-party rating of program’s students

7. Ratings of in-service, clinical, and PDS

teaching


Inventory: Status of Evidence from measures and indicators for TEAC Quality Principle I

Type of Evidence Available and in the Brief Not Available and Not in the Brief

Note: items under each category are

examples. Program may have more or

different evidence

Relied on

Reasons for including the results in the

Brief

Location in Brief

Not Relied on

Reasons for not

relying on this

efficiency

Location in Brief

For future use

Reasons for including in

future Briefs

Not for future use

Reasons for not including in future

Briefs

8. Ratings, by cooperating teacher and

college/university supervisors, of

practice teachers’ work samples.

Students can demonstrate

knowledge and professional

skills

4,5

Rates

9. Rating of completion of courses and

program

Support professional

knowledge 1

10. Graduates’ career retention rates

Most of our graduates’ occupy

leadership positions before entering the program; consequently is not a

good measure of program

accomplishments.

11. Graduates’ job placement rates See above

12. Rates of graduates professional advanced

study

13. Rates of graduates’ leadership roles

Emphasis is placed in

developing job abilities and

skills in program SOED

graduates are expected to

make contributions as

educational leaders. Some

kind of follow up will be

included in future surveys.

14. Rates of graduates’ professional service activities

Service is one of the

cornerstones of SOED graduates. Data regarding

service will be included in

future surveys


Case Studies and alumni competence

15. Evaluations of graduates by their own

pupils

Graduates occupy leadership roles;

consequently it is very difficult to

obtain reliable data.

16. Alumni self-assessment of their

accomplishments

Data files were damaged during the huricane.

Graduate over perceptions

are essential for program

improvement

17. Third party professional recognition of

graduates (e.g., NBPTS)

Efforts in follow up surveys

will be made to obtain this

data but it has proven to be very difficult to obtain in the

past.

18. Employers’ evaluation of the program’s

graduates

Necessary for program

improvement. Must be

incorporated in future

surveys.

19. Graduate’s authoring of textbooks,

curriculum materials, etc.

Data will be collected in

future surveys

20. Case studies of graduates’ own pupils’

learning and accomplishments

Useful for program

improvement. Will be

included in EDUC 604

Comprehensive exams

21. EdD comprehensive exams

Demonstrate a

comprehensive perspective

of knowledge and skills

acquisition

3, 5, 7,

8

universidad del turabo school of education educational ......of program course content with a...

Documents