quality enhancement review august 2017 … · in fall 2015, zhang joined the program at the rank of...

Quality Enhancement Review

August 2017

Measurement & Statistics Program

Department of Educational Psychology and Learning Systems

College of Education

Florida State University

1

Quality Enhancement Review: Self-Study Questions (2017-2018)

Unit Overview:

Cover Page Provide the following: unit name, unit website address, college, names of unit leadership,

unit bylaws web address, outside accreditation information, if applicable, – organization

and date of last reaffirmation, and term of last QER.

Unit name: Measurement and Statistics (M&S)

Unit website address: http://education.fsu.edu/degrees-and-

programs/measurement-and-statistics

College: College of Education

Names of unit leadership: Betsy J. Becker (department chair); Yanyun

Yang (program coordinator for Measurement & Statistics)

Department bylaws web address: http://education.fsu.edu/wp-

content/uploads/2015/05/EPLS-Bylaws-2016-05-06.pdf

Outside accreditation information: N/A

Term of last QER: Fall 2010

http://education.fsu.edu/degrees-and-programs/measurement-and-statistics

http://education.fsu.edu/degrees-and-programs/measurement-and-statistics

http://education.fsu.edu/wp-content/uploads/2015/05/EPLS-Bylaws-2016-05-06.pdf

http://education.fsu.edu/wp-content/uploads/2015/05/EPLS-Bylaws-2016-05-06.pdf

2

Table 1 – Degree Program Overview Complete the last columns of Table 1 and insert behind this tab.

Table 1 provides information related to degree programs offered (divided by

major), required hours, limited access (if applicable), current term enrollment by

degree program, diversity information by degree program, and the number of

graduates by degree program for past five years (total). Units will provide name

of faculty coordinator for each degree program and comment on the coordinators

academic credentials and qualifications to serve in that role.


Educational Psychology & Learning Systems | Measurement & Statistics Program

Table 1: Degree Program Overview

Overview

Major Name Academic Plan Code CIP Code Degree Offered

Required

Hours Total International Minority*

Measurement/Statistics - EDS MSURSTATES 130603 Specialist's (not published) N/A 0 0 0 1

Measurement/Statistics - MS MSURSTATMS 130603 Master of Science 33 3 2 1 16

Measurement/Stats-MS/Thesis MSURSTATMT 130603 Master of Science/Thesis 33 1 1 0 3

Measurement/Statistics - PHD MSURSTATPD 130603 Doctorate 78-106 20 18 2 18

Program Total 24 21 3 38

Measurement/Statistics - Centificate *** 19 39 14 2 85

*Minority includes: American Indian/Native Alaskan, Asian, Black, Hispanic, Native Hawaiian/Pacific Islander, and Two or More Races.

**Degrees Awarded represent a 5-year total from 2011-12 through 2015-16.

***These figures were generated from departmental records.

Source: FSU Degree Program Inventory; Fall 2016 Student Instruction Files (SIFP) for Enrollment; Campus Solutions Warehouse - Term Statistics for Degrees Awarded.

Faculty Coordinator

Major Name Academic Plan Code CIP Code Degree Offered Comments

Measurement/Statistics - EDS MSURSTATES 130603 Specialist's (not published)

Measurement/Statistics - MS MSURSTATMS 130603 Master of Science

Measurement/Stats-MS/Thesis MSURSTATMT 130603 Master of Science/Thesis

Measurement/Statistics - PHD MSURSTATPD 130603 Doctorate

Measurement/Statistics - Certificate ***

***These figures were generated from departmental records.

Source: FSU Degree Program Inventory; Department of Educational Psychology & Learning Systems

2009, except Fall 2014-Spring 2015 when she

Paek has served as faculty coordinator since 2011

Yang, Yanyun

Paek, Insu

Yang received Ph.D in Educational Psychology.

She has served as faculty coordinator since Fall

Yang, Yanyun was on sabbatical leave.

Degrees Awarded

(5-Year Total)**

Fall 2016 Enrollment

Faculty Coordinator

Yang, Yanyun

Yang, Yanyun

3

Table 2 – Faculty Information

Complete the last column of Table 2 and insert behind this tab.

Table 2 presents unit faculty information including degree/major, year,

institution, and tenure status. Units will provide information related to affiliation

by degree program. Multiple degree program affiliations may be listed for faculty

members.



Table 2: Faculty Information

Ranked Faculty

Last Name First Name

Degre

e Degree University Job Description Tenure FSU Hire Gender Race Program Affiliation

Almond Russell PhD Harvard University Asoc Professor 9 Mo SAL Tenured 8/9/2010 Male White Measurement & Statistics

Becker Betsy PhD University of Chicago Professor 9 Mo SAL Tenured 8/9/2004 Female White Measurement & Statistics

Paek Insu PhD Univ. of California-Berkeley Asoc Professor 9 Mo SAL Tenured 8/9/2010 Male Asian Measurement & Statistics

Yang Yanyun PhD Arizona State University Asoc Professor 9 Mo SAL Tenured 8/9/2007 Female Asian Measurement & Statistics

Zhang Qian PhD University of Notre Dame Asst Professor 9 Mo SAL On Track 8/10/2015 Female Asian Measurement & Statistics

Source: HCM - Workforce - Employee Job database

Additional Faculty and Instructors

Last Name First Name

Degre

e Degree University Job Description Tenure FSU Hire Gender Race Program Affiliation

Binici Salih PhD Florida State University Teaching Faculty I V. in Lieu Not on Track 1/7/2008 Male White Measurement & Statistics

Olgar Suleyman PhD Florida State University Teaching Faculty I V. in Lieu Not on Track 5/15/2017 Male White Measurement & Statistics

Source: HCM - Workforce - Employee Job database

4

1. Major Changes/Unit Strengths/Weaknesses

Summarize in bullet format:

a. Major changes in the unit since the last QER (e.g., new/halted/terminated degree

programs, unusual turnover in faculty or changes in leadership positions such as

program coordinators, new/renovated facilities, etc.).

b. The unit’s major strengths and weaknesses.

Measurement and Statistics (M&S) has seen just a few changes since the last

review:

In Fall 2010, M&S hired Almond and Paek who are specialists in

measurement and educational assessment. Both are currently at the

rank of associate professor with tenure.

In Fall 2015, Zhang joined the program at the rank of assistant professor.

With five faculty members in the program, we are able to offer not only

service statistics/measurement courses, but also special-topics courses for

advanced doctoral students in our program.

Program strengths.

The academic and social climate in the Measurement & Statistics

program is excellent, both for faculty and students. Our mission continues

to include both service to the college and FSU, and a focus on advancing

techniques in educational measurement and statistics and sharing that

scholarship in courses for our own students.

The courses offered by the M&S program, especially the introductory

statistics sequence (EDF 5400, EDF 5401, EDF 5402), advanced classes in

statistics (EDF 5406, EDF 5409, and EDF 7489), and the introductory

measurement class EDF 5432, have been attractive to students from

various disciplines across campus.

All faculty members have degrees from very high research active (RU/VH)

universities (see Table 2), and from strong departments within those

schools. Senior faculty colleagues have high levels of professional

recognition internationally, and serve the profession in many ways. A few

examples include that Becker was elected as a Fellow of the American Educational Research Association in 2013, and was also named a

Distinguished Research Professor by FSU in 2016. Almond was

nominated for the Outstanding Contribution to Research in Cognition and Assessment, American Educational Research Association, Cognition and

5

Assessment Special Interest Group in 2015 and received an honorable

mention.

Faculty are skilled teachers, as shown in the teaching evaluations

(discussed in Question 11).

Ongoing collaborations on projects among faculty within and outside of

M&S provide a collaborative context. Almond has collaborated on

research on Game-based Assessment with Shute and Ke from the

Instructional Systems and Learning Technologies program, serving as co-

Investigator on several grants. Almond published a well-received book

Bayesian Networks in Educational Assessment with former colleagues

from Educational Testing Service (Robert Mislevy, Linda Steinberg,

Duanli Yan, and David Williamson), based on research which had

previously won the 2000 NCME award for Outstanding Scientific or

Technical Contribution to Educational Measurement. Becker has

consistently been funded as a PI by the National Science Foundation to

conduct projects on research synthesis, and has collaborated with

researchers in fields such as nursing and sport psychology. Yang worked

as key personnel on an IES-funded project regarding performance

assessment. Faculty collaborate with various testing companies including

ETS, Pearson, WestEd, Behavior Research and Technology, and the

Dynamic Learning Maps consortium; and use their contacts from that

consulting to help students find research projects. Students are actively

involved in faculty research, with multiple research groups open to all

students and not just the advisees of the faculty leader.

People like each other and all members do their share to make sure the

program runs well. This extends to both the program and across

programs in the department. One consequence of this collegiality is that

faculty can literally step outside their door and find a colleague to discuss

problems with teaching, research, or service obligations. Another

consequence is that the mentoring of new faculty members and even

students is seen as a shared responsibility with whoever is available and

best able to help stepping up when the need arises.

The program has a very close relationship with the Florida Department of

Education (FLDOE). The M&S program has been running a psychometric

internship program and collaborating with FLDOE in support of high

quality psychometric analyses for the Florida Comprehensive Assessment

Test (FCAT) and Florida Standards Assessments (FSA). Three to four

M&S graduate students are typically selected to work as FLDOE

psychometric interns for roughly three years each.

6

Students come from diverse backgrounds. Most students in M&S are

international – coming primarily from China, South Korea, Saudi Arabia,

and Turkey. As shown in the table below, the GRE score profile in M&S is

a bit uneven, due to the much higher quantitative scores and lower verbal

scores of the quantitatively talented international student pool. Almost all

applicants in M&S come from relevant undergraduate/masters’ programs

including educational psychology, psychology, statistics, math, and math

and science education.

Mean GRE Scores for Admitted Graduate Students in 2010-2016

Prior to 8/1/11 After 8/1/11

Verbal Quantitative Verbal Quantitative

Master's

(n=12)

497 668 149 157

Doctoral

(n=25)

402 721 148 161

Program weaknesses

A major concern in the program is finding financial support for students.

In fact, all programs in the department have this concern. The

department, college, and university offer some financial awards to high-

achieving and high-scoring students, but many graduate students are

forced to look for other resources such as loans and grants, teaching

assignments, research assistantships, or outside employment. Some

students who are in the dissertation-writing stage work as full-time

employees outside of campus (e.g., at the Florida Department of

Education). This considerably prolongs their dissertation progression and

time to graduation. Lack of financial support for prospective students has

become a serious issue for recruiting students in the past few years (more

on this issue in Question 13 and Question 23).

The enrollment for some of our courses has been erratic during the past

few years. This is particularly true for EDF 5400, EDF 5401, and EDF

5406. The enrollment ranges from very small (e.g., 6) to big (e.g., 42). The

advantage of having small class sizes is that the instructor-student

interaction is guaranteed. However, the erratic enrollment across

semesters brings some unpredictability into our course offerings. For

example, one section of EDF 5400 had to be canceled in Fall 2016, which

may have impacted some students’ progression toward their degrees. This

can occur when departments that usually send their students to take our

classes decide to offer their own classes on statistics.

7

The program offers no undergraduate courses. This means that

graduates who are trained for academic careers do not get a chance to

practice teaching "officially" or get teaching experience on their vitas. We

have tried to address it by allowing students to conduct review sessions

or to give guest lectures in some classes.

8

2. Academic Program History Provide a brief history of the unit and its component academic programs. Highlight key

events contributing to its current academic offerings and organizational structure.

The program in Measurement and Statistics (M&S) is situated within the

program area known as Educational Psychology, in the Department of

Educational Psychology & Learning Systems (EPLS) in the College of

Education. EPLS has currently three program areas: Educational

Psychology, Instructional Systems and Learning Technologies, and

Psychological and Counseling Services. Other majors within Educational

Psychology are Learning & Cognition and Sport Psychology.

EPLS is a department with a long history. In 1948, the department was

called simply 'Education', and it offered courses in measurement, statistics,

and theories of learning – fundamental offerings of two of the current

Educational Psychology majors. Over the years, the department expanded to

include other foundational areas of education, such as evaluation, research

methods, and history and philosophy of education. By the early 1960s, the

department was reconstituted and renamed 'Educational Research & Testing'

and faculty in history and philosophy of education moved to a separate

department.

In 1969, a clear distinction was made among three degree programs:

Educational Evaluation and Measurement, Educational Psychology, and

Instructional Systems Design. Educational Evaluation and Measurement

focused on the tools of research, Educational Psychology on the theoretical

foundations of learning and development, and Instructional Systems Design

on the application of both learning theory and research tools to the

development of empirically-based instructional interventions.

During a reorganization of the college in 1978, the department again

annexed programs in educational foundations and was renamed 'Educational

Research, Development, and Foundations'. This structure remained in place

until the mid-1980s when the history and philosophy of education group

moved to a separate department of Educational Foundations. Our

department then became known as 'Educational Research'. In 1992, the Sport

Psychology major joined the department, and consolidation of majors resulted

in two program areas in the department: Instructional Systems, and

Educational Psychology (EP). EP consisted of majors in Measurement &

Statistics, Program Evaluation, Learning & Cognition, and Sport Psychology.

Program Evaluation has since moved to another department.

9

In 2002 a third program area joined the department: Psychological Services

in Education (PSE, now known as Psychological and Counseling Services),

and in 2009 faculty in Rehabilitation Counseling joined the counseling group.

Students from all of these program areas enroll in courses offered by M&S. In

2002 the department was renamed 'Educational Psychology and Learning

Systems' to better reflect the current mission and composition of programs.

10

Curriculum:

3. Degree Program Description Provide a description of the degree programs and majors offered by the unit as outlined in

Table 1.

For undergraduate programs, please provide complete links to the Academic Program

Guide found at: http://www.academic-guide.fsu.edu.

The program in M&S offers only graduate degrees, thus has no undergraduate majors.

For graduate programs, please provide complete information regarding courses

(including titles), research or other requirements. Units with complete information

published on a website may submit the website URL.

a. What aspects of the curricula distinguish them from similar programs around the

country?

b. How do these programs contribute to FSU’s mission

(https://www.fsu.edu/about/mission_vision.html)?

The program in Measurement and Statistics (M&S) offers both Master’s of

Science and Doctor of Philosophy degrees, as well as a certificate in

Measurement and Statistics.

The M&S Certificate Program. The M&S certificate program enables

graduate students in non-M&S programs to be equipped with theoretical and

practical knowledge of psychometrics and statistical analysis skills in social

science areas including education, without the full commitment to a 30+

credit master's degree. Students must apply to the M&S certificate program

and be admitted to be qualified as M&S certificate students. The M&S

certificate is given to admitted M&S certificate students who complete a

minimum of 19 credit hours of the M&S program courses with "B" or higher

grades.

Required Courses

EDF 5401 General Linear Models Applications (4 hrs)

EDF 5402 Advanced Topics in ANOVA (3 hrs)

EDF 5432 Measurement Theory I (3 hrs)

Plus any three of the following courses

EDF 5404 Bayesian Data Analysis (3 hrs)

EDF 5406 Multivariate Analysis Applications (3 hrs)

EDF 5409 Causal Modeling (3 hrs)

EDF 5431 Classroom Assessment (3 hrs)

EDF 5434 Measurement Theory II (3 hrs)

EDF 5435 Theory of Scaling and Equating (3 hrs)

11

EDF 5448 Scale/Instrument Development (3 hrs)

EDF 5484 ED Data Analysis (3 hrs)

EDF 7418 Multilevel Modeling (3 hrs)

EDF 7489 Meta-Analysis (3 hrs)

EDF 6937 (Advanced seminars)

The M&S Master’s Program. The master’s degree in Measurement and

Statistics provides a minimal theoretical and research background in the

domain, but its emphasis is on building preliminary, mainly applied, skills in

measurement and statistical analysis. This degree can complement a higher

degree in a related discipline, such as Educational Leadership and Policy

Studies, Public Policy, Learning and Cognition, or Instructional Systems and

Learning Technologies. Students may take one of two tracks toward the

degree: thesis track or non-thesis track.

The master’s thesis track requires:

A minimum of 30 credit hours of coursework, at least 18 of which must

be letter-graded (e.g., A, B, C; in particular, these credits cannot be

earned via directed independent study which is graded U/S).

A minimum of 6 thesis hours.

Passing the thesis prospectus defense.

Passing the thesis defense.

The master’s non-thesis track requires:

A minimum of 32 credit hours of coursework, at least 21 of which must

be letter-graded (e.g., A, B, C); in particular, these credits cannot be

earned via directed independent study which is graded U/S).

Courses taken more than 7 years prior to the comprehensive exam may

not be counted towards these numbers.

Passing grade on the Comprehensive Exam.

To be eligible for conferral of a master's degree, students must have a

cumulative grade point average (GPA) of at least 3.0 in formal graduate

courses. No course hours with a grade below "C-" will be credited to the

graduate degree, and all grades in graduate courses except those for which

grades of "S" or "U" are given or those conferred under the provision for

repeating a course will be included in the GPA. In addition students must

earn a grade of "B" or better in all required M&S graduate courses. Grades

earned at another institution cannot be used to improve a GPA at Florida

State University.

12

The required and recommended courses for the master’s degree in

Measurement and Statistics are:

Required Courses

EDF 5400 Descriptive/Inferential Statistics Applications (4 hrs)






EDF 5481 Methods of Educational Research (3 hrs)

EDF 5916 Research Proposal Writing (1 hr extra with EDF 5481for thesis-

track M.S. students)

EDP 5935 Educational Psychology (3 hrs)

EDF 5971 Thesis (Thesis track)

EDF 8966 Comprehensive Exam (Non-thesis track)

Recommended Courses






EDF 6937 (Advanced seminars)

The M&S Doctoral Program. The Ph.D. program in Measurement and

Statistics emphasizes both advanced instruction and direct research

experience with measurement and applied statistics issues. Its requirements

are slightly less structured than those for the master’s degree. This enables

students who enter the program with varying backgrounds and degrees of

knowledge to tailor their programs to their own interests and needs. Ph.D.

students who do not hold a master’s degree in M&S are encouraged to take

the course work required for the M.S. to ensure that they acquire a suitable

foundation. Students with a Master’s degree in a related field or who have

graduate work from another accredited university typically consult their

faculty advisor about waiving some of the courses that are required for the

Ph.D. program. The principal requirements for the Ph.D. degree are:

Completing or waiving required coursework.

Passing the qualifying examination.

Passing the preliminary examination.

13

Passing the dissertation prospectus defense.

Passing the dissertation defense.

To be eligible for conferral of a doctoral degree, the student must earn a

cumulative grade point average of at least 3.0 in formal graduate courses. No

course hours with a grade below "C-" will be credited on the graduate degree;

all grades in graduate courses except those for which grades of "S" or "U" are

given or those conferred under the provision for repeating a course will be

included in computation of the average. In addition, students must earn a

grade of "B" or better in all M&S required and core graduate courses. Grades

earned at another institution cannot be used to improve a GPA or eliminate a

quality point deficiency at Florida State University. In addition to coursework

and the dissertation, doctoral students are expected to meet the scholarly

engagement requirement of Florida State University (see also Question 15c).

The required and recommended courses for the doctoral degree in

Measurement and Statistics are:

Required Courses

EDF 5400 Descriptive/Inferential Statistics Applications (4 hrs)




EDF 5409 Causal Modeling (3 hrs)



EDF 5434 Measurement Theory II (3 hrs)

EDF 5435 Theory of Scaling and Equating (3 hrs)


EDF 5481 Methods of Educational Research (3 hrs)

EDF 5916 Research Proposal Writing (1 hr extra with EDF 5481)

EDP 5935 Educational Psychology (3 hrs)


EDF 6980 Dissertation (24 hrs)

Two Courses from one of the following programs

Program Evaluation

Instructional Systems and Learning Technologies

Learning & Cognition (excluding already required courses)

Psychology

The chosen courses must represent an area of application of measurement and

14

statistics, and not merely be methodological courses.

Recommended Courses from M&S



EDF 6056 Bayesian Networks (3 hrs)

EDF 6057 Large-scale Assessment (3 hrs)


EDF 6937 Advanced Meta-Analysis (3 hrs)

EDF 6937 Advanced Structural Equation Modeling (3 hrs)

EDF 6937 Bayesian IRT (3 hrs)

EDF 6937 Survey Sampling (3 hrs)

EDF 6937 Missing Data Analysis (3 hrs)

EDF 6937 Longitudinal Data Analysis (3 hrs)

EDF 6937 Multidimensional IRT (3 hrs)

EDF 6937 Experimental/Quasi Experimental Design (3 hrs)

Recommended Courses from the Statistics Department

STA 5106 Computational Methods in Statistics I (3 hrs)

STA 5207 Applied Regression Methods (3 hrs)

STA 5238 Applied Logistic Regression (3 hrs)

STA 5325 Mathematical Statistics (3 hrs)

STA 5323 Introduction to Mathematical Statistics (3 hrs)

STA 5326 Distribution Theory & Inference (3 hrs)

STA 5440 Introductory Probability I (3 hrs)

STA 5507 Applied Nonparametric Statistics (3 hrs)

STA 5707 Applied Multivariate Analysis (3 hrs)

To answer item 3a, “What aspects of the curricula distinguish them from

similar programs around the country?”, we note that our curricula are

distinct from those of similar programs around the country in at least two

ways. First, all five faculty members have their own research specialties,

allowing us to offer a diversity of advanced special topics courses to graduate

students. Similar programs around the country may offer some of these

courses but not all. These courses include Advanced Meta-analysis, Bayesian

IRT, Bayesian Networks in Educational Assessment, Longitudinal Data

Analysis, Measurement Invariance Analysis, Missing Data Analysis, and

Scaling and Equating.

15

Second, our program has a very close relationship with the Florida

Department of Education (FLDOE). At any point in time three to four

selected M&S students will work as FLDOE psychometric interns. The

FLDOE interns also have plenty of opportunities to experience how the

psychometric and statistical methods learned through the M&S curricula are

applied in large-scale assessments such as the Florida Standards

Assessments (FSA) and Florida Comprehensive Assessments (FCAT). The

M&S program has been collaborating with FLDOE through the psychometric

program for operational work and research projects related to large-scale

assessments for more than the past fifteen years.

To address item 3b, “How do these programs contribute to FSU’s mission”, we

first note the recognition brought to the university thanks to the high-quality

research, strong scholarly productivity, and national and international

prominence of the faculty. Holding important committee leadership roles for

national agencies and associations, serving on professional association

committees and as editorial board members for top journals, our faculty

increase the visibility of Florida State University among education scholars

worldwide.

A second key contribution is our service, via the wide subscription to our

courses from students from across the university, and our faculty members'

service as outside representatives on thesis and dissertation committees

university-wide.

One course, EDF 5400 Descriptive/Inferential Statistical Applications, is used

as an introductory statistics course for many departments both inside and

outside of the College of Education. This course is designed to address one of

the core missions of the University: developing the critical thinking skills of

students. EDF 5400 specifically targets students’ ability to reason with data

via a number of case studies where students work in small groups to

interpret the results of a statistical analysis. The capstone activity is a series

of lab activities where students perform basic statistical analyses on real data

supplied by the instructor, and write up the results as if they were to be

presented in a research paper. This valuable experience, which usually

happens early in the students’ graduate careers, prepares them for later

technical writing in their dissertation and thesis.

M&S advanced courses are also of wide appeal across the university due to

the teaching skill and the national and international expertise of M&S

faculty. Courses like Multivariate Analysis Applications (EDF 5406), Causal

Modeling (EDF 5409), Multilevel Modeling (EDF 7418), and Meta-analysis

(EDF 7489) routinely fill and draw from diverse programs in other colleges

16

such as Communications, Public Policy, Nutrition and Family Studies, and

the like.

Finally, due to the wide appeal of our courses, M&S faculty members

routinely receive requests to serve as methodological experts on dissertation

committees around campus. This often means taking on the role of University

Representative. Our faculty sometimes carry heavy loads in this regard.

17

4. Curriculum Review When did the faculty last conduct a comprehensive review of the curriculum in each of

its degree programs and majors? What was the outcome of those review processes?

a. If a unit is accredited by an outside organization, please summarize the results of

the latest reaffirmation/review.

b. How are curriculum decisions made within the unit? Who has final authority?

The unit is not accredited by an outside organization.

The last time the M&S faculty conducted a formal comprehensive review of

the curriculum was Fall 2010, though informal reviews occur with each

decision to create a new course or seminar. The last formal review process

identified two issues around course offerings. First, staffing shortages had

led to some classes not being offered for serval years (e.g., EDF 5448

Instrument and Scale Construction; EDF 5410 Nonparametric Statistics).

Second, enrollments in the service courses, especially in the introductory

statistics courses, were very large (up to 62 in EDF 5400). These two issues

have partially been resolved with hiring of Almond and Paek in Fall 2010

and Zhang in 2015. We now are able to not only offer classes that we were

not able to offer previously (EDF 5448), and to slightly lower the enrollment

in the introductory statistics courses, but we can now also offer a diversity of

advanced special topics courses to graduate students. However, the course

EDF 5410 Nonparametric Statistics has still not been offered due to a lack of

staff who specialize in this area.

The development of new courses and updating and changing of existing

courses are done by the faculty members within the M&S program, and then

submitted to curriculum committees (through department, college, and

university levels). Without formal review, faculty members may offer

seminar courses covering recent research and theory; topics offered have

included Missing Data Analysis, Bayesian Networks in Educational

Assessment, Advanced Meta-Analysis, and Advanced Structural Equation

Modeling. As these courses mature, they can change into more formal

courses, which are then reviewed by the department, college, and university

curriculum committees.

18

5. Student Learning Outcomes Insert Table 3.

Table 3 includes the Student Learning Outcomes and Assessment Methods for all

degree programs in the unit.

a. How does the unit utilize the results in the Institutional Effectiveness Portal to

improve student learning outcomes?

b. What learning deficits have been identified through this process? How were those

addressed?

Table 3 displays the Institutional Effectiveness (IE) outcomes and related

assessment methods for both master's and doctoral programs in M&S; these

are repeated below. The first two are student learning outcomes and the third

is a program outcome. These learning goals and assessment methods are

arrived at through discussion and consensus.

Yearly examination of SACS goals (early in each Fall semester) contributes to

the process of ongoing quality review. Seeing whether we have achieved our

results (or not) has led to changes, for example in the nature and content of

our colloquium series, in how faculty interact with master’s students, and the

like. By prioritizing as goals some of our most important values for student

learning and behaviors, we keep them “front and center” in our program’s

events and structure. Below we review our goals and describe some of the

changes we have made as a part of the IE process.

Master’s program.

Goal 1: Students in the master's program will attend the program's

colloquium series. The goal is for students to participate in conversations

and enter into debate about research and methodologies relevant to

measurement and statistics, both as presenters and attendees. Criterion

for success: 80% of students will attend the sessions during fall & spring

terms of each year, with the exception of excused absences.

Goal 2: Students in the master's program who want to be involved in

research activity will be involved in research projects, either on their own

or in collaboration with faculty. The goal is for students to get as many

opportunities as possible to apply their measurement and statistics skills

in authentic research settings. Criterion for success: 50% of all master's

students who are fully enrolled in the M&S major and who want to

participate in research will be involved in research activities at some

point during the academic year.

19

Goal 3: One of the key ways we can recruit doctoral students is from the

master's program. To accomplish this we should recruit master's

students who express an interest in research, even if they may not be

ready to commit to doctoral study immediately. Our goal is to identify

and enroll more master's students who will eventually enter the doctoral

program. Criterion for success: Two master's students will apply for

doctoral study; two additional master's students with doctoral study

ambitions will be recruited in the academic year.

The first goal was not met in most years during 2010-2016. However in all

years, if we exclude students who were dual degree candidates (i.e., enrolled

in another doctoral program as well as our master's program) or who were

working full time, we missed the goal by only a few percent. In every year, the

student coordinators of the colloquium met to discuss more interesting topics

that would potentially encourage better attendance. The colloquium

coordinator sent emails to students reminding them of every event. This was

emphasized at the orientation colloquium and also to potential students when

they were considering applying to the program. We also asked students to

inform us of why they were not attending colloquium and to provide a reason

for their absence.

The second goal was met in all years except in 2015-2016. Since 2009, we

started to encourage all masters’ students to join a research group (e.g., R and

BUGS user group, meta-analysis research group) and to begin reading

journals in the field, and have continued this practice to this date. The plan

was to give all students the chance to be involved in research whether they

expressed an interest and actively pursued the idea or not. Major advisors

discussed research interests with their students, monitored ongoing research

groups for effectiveness, and provided support that seemed needed. However,

some students chose to not participate in research projects or research

groups. Most of them were in the coursework track and either did not express

interest in research, or were not interested in pursuing doctoral study in

measurement and statistics. Considering these characteristics of students in

our master’s program, in 2016 we decided to lower the criterion for success

from 100% to 50%.

The third goal was met in all years. However, the numbers speak to a

problem further up the pipeline. We generally have two sources of master's

degree students: (1) new recruits and (2) students with dual enrollment with

other programs in the College of Education or other colleges. The latter

students are typically already pursuing PhD degrees in their other field of

study. Many of the new recruits are interested in continuing on with a PhD

degree. The problem is not with encouraging the existing students but rather

with recruiting more master's candidates. As we discussed in addressing

20

Question 1b, we have encountered difficulties in recruiting new students. In

the 2017 class, all four students in the master’s program are dually enrolled.

We will reach out to various undergraduate majors, such as Statistics and

Psychology, which provide good preparation for a future degree in

Measurement & Statistics.

In addition to the observed learning goals, master’s students who select a

coursework option are required to take comprehensive exams upon

completion of their coursework toward the master’s degree. The exam consists

of two components: in-class exams and an oral defense. Master’s students

selecting the thesis option choose a topic in the area on which to conduct

research. They are required to pass both prospectus and final thesis defenses

by consensus of a committee with a minimum of 3 faculty members.

Doctoral program.

Goal 1: Students in the doctoral program will join professional

organizations such as, but not limited to, the Florida Educational

Research Association, American Educational Research Association, the

National Council for Measurement in Education, and the American

Statistical Association. The goal is for students to gain access to the

greater professional community and research journals in order to explore

their interests and integrate themselves into the professional community.

Criterion for success: 100% of students beyond the first year should

belong to at least one of these organizations.

Goal 2: Students in the second year of the doctoral program (and beyond)

will attend professional conferences such as, but not limited to, the

meetings of the Florida Educational Research Association (FERA), the

American Educational Research Association, the National Council for

Measurement in Education, and the American Statistical Association.

The goal is for students to have the opportunity to interact face to face

with colleagues in their field and have opportunities to learn about,

discuss, and present on cutting edge research and methodologies.

Criterion for success: 65% of students in the second year and beyond will

be involved in conference presentations each year.

Goal 3: The program wants to identify faculty in other programs on

campus who have strong interest and skills in areas related to those of

program faculty. Those faculty will be invited to be listed as affiliated

faculty, and to be involved in program activities such as the program

colloquium, student committees etc. The goal is to provide students with

more opportunities to interact with faculty and more opportunities to

recognize and explore the application of measurement and statistics

21

across diverse disciplines. Criterion for success: Listing five affiliated

faculty during the academic year. The program coordinator will keep

track of affiliated faculty members.

The first goal was not met in all years. Most students reported being

members of professional associations, but we missed the criterion of 100% to

varying degrees across years. We sometimes reminded students to join

associations, but we do not want to impose on their right to protect their own

economic status, since costs of memberships may be a contributing reason for

their not joining.

The second goal was met in all years except in 2012-2013 and 2015-2016.

Faculty members have actively encouraged students to participate as authors

and conference presenters. In addition, our faculty members' grants have

provided great opportunities for students to be involved in research projects,

which help increase students' conference involvement. In 2012-2013 and

2015-2016, most of those who did not attend professional conferences were

either working full-time off campus, attempting preliminary exams, or

actively working on their dissertation projects. We will keep our goal for the

coming years at the level of 65% participation. It does not make sense for all

students to be authors every year as the newer students may not be ready to

participate in this way. Students' major advisors will actively encourage them

to participate in conferences. We will also advertise appropriate conference

deadlines through our student mailing lists.

The third goal was met in all years. We will continue to keep an eye out for

new hires in other programs who could help us offer courses. But currently we

have enough formal affiliates to cover our courses. We will have more

informal affiliations to help students place Measurement and Statistics more

broadly in the discipline of Education and to broaden the kind of input

students can get in their research.

Scholarly engagement.

Starting from Fall 2016, FSU and the M&S major have implemented a new

set of Scholarly Engagement requirements to encourage doctoral students to

be more engaged in professional development and research. We have designed

our requirements to enhance and mesh with our Institutional Effectiveness

goals. Specifically, doctoral students in the first two years of study should

attend a professional conference that is related to their program of study (e.g.,

American Educational Research Association, American Psychological

Association, American Statistical Association, or Florida Educational –

FERA). Doctoral students in their third year of study and beyond should give

22

one or more presentation(s) related to their program of study, such as being

an (co)-author in a professional conference. Alternatively, they could choose to

meet the requirement by being involved in preparing, submitting, or

publishing research paper(s) in scholarly outlets (e.g., journals, books) as

authors or coauthors. The newly implemented scholarly engagement

requirement matches with the first two doctoral IE goals (see Questions 5 and

15c).

In addition to the above three learning goals, doctoral students are required to

complete a qualifying review (including a program of study) by the end of the

first two semesters in residence. The student and the initial advisor select a 3-

person review committee. The supervisory committee and the student meet

afterwards to review and discuss how the proposed program will help the

student to meet their goals.

College of EducationEducational Psychology & Learning Systems | Measurement & Statistics ProgramTable 3: Student Learning Outcomes

Masters in Measurement & Statistics

Outcome Type Outcome Assessment & Evaluation ProcessStudent Learning Students in the master's program will attend the program's colloquium series. The goal is for

students to participate in conversations and enter into debate about research and methodologies relevant to measurement and statistics, both as presenters and attendees.

Criterion for success: 80% of students will attend the sessions during fall & spring terms of each year, with the exception of excused absences. All students are expected to attend the colloquium series, which meets every other week. However, because some students work full time, there will be occasions when work requirements overrule attendance at the colloquium. Also, illness or family emergencies clearly override the requirement to attend. A sign-in sheet is used to take attendance. (We may need to consider whether the goal is sensible for students who are dual enrollees because they also have responsibilities to their primary programs that likely, and probably should, take precedence over ours.)

Student Learning Students in the master's program who want to be involved in research activity will be involved in research projects, either on their own or in collaboration with faculty. The goal is for students to get as many opportunities as possible to apply their measurement and statistics skills in authentic research settings.

Criterion for success: 50% of all students who are fully enrolled in the M&S major and who want to participate in research will be involved in research activities at some point during the 2016-2017 year. At the end of the academic year, students will submit a written report of their activities during the year. The coordinators of the annual evaluation will summarize information on research participation.

Program Outcome One of the key ways we can recruit doctoral students is from the master's program. To accomplish this we should recruit master's students who express an interest in research, even if they may not be ready to commit to doctoral study immediately. Our goal is to identify and enroll more master's students who will eventually enter the doctoral program.

Criterion for success: Two masters student will apply for doctoral study; two additional masters students with doctoral study ambitions will be recruited in the 2016-2017 year. We are setting this goal low and have kept it the same as in the previous year. Many masters students enter the program with a terminal degree in mind, so it may be unreasonable to expect those currently enrolled to apply for doctoral study. The program coordinator will collect the data from the applicant database and via contact with applicants.

Source: FSU Institutional Effectiveness Portal, 2016-17.

College of EducationEducational Psychology & Learning Systems | Measurement & Statistics ProgramTable 3: Student Learning Outcomes

Doctorate in Measurement & Statistics

Outcome Type Outcome Assessment & Evaluation ProcessStudent Learning Students in the doctoral program will join professional organizations such as, but not limited

to, the Florida Educational Research Association, American Educational Research Association, the National Council for Measurement in Education, and the American Statistical Association. The goal is for students to gain access to the greater professional community and research journals in order to explore their interests and integrate themselves into the professional community.

Criterion for success: 100% of students beyond the first year should belong to at least one of these organizations. At the end of 2016-2017 year, students will submit their resume as part of their written report. We will also ask students to list professional memberships in their annual evaluation. The program coordinators will summarize information on professional memberships.

Student Learning Students in the second year of the doctoral program (and beyond) will attend professional conferences such as, but not limited to, the meetings of the Florida Educational Research Association, the American Educational Research Association, the National Council for Measurement in Education, and the American Statistical Association. The goal is for students to have the opportunity to interact face to face with colleagues in their field and have opportunities to learn about, discuss , and present on cutting edge research and methodologies.

Criterion for success: 65% of students in the second year and beyond will be involved in conference presentations in the 2013-2014 year. Involvement can include being part of a submitted proposal even if it was not accepted, and also of course will include being an author and presenter or a co-author on an accepted presentation. At the end of the 2013-2014 year, students will submit their resumes as part of their written report. The program coordinator will summarize information on their conference attendance.

Program Outcome The program wants to identify faculty in other programs on campus who have strong interest and skills in areas related to those of program faculty. Those faculty will be invited to be listed as affiliated faculty, and to be involved in program activities such as the program colloquium, student committees etc. The goal is to provide students with more opportunities to interact with faculty and more opportunities to recognize and explore the application of measurement and statistics across diverse disciplines.

Listing five affiliated faculty during 2016-2017 year. The program coordinator will keep track of affiliated faculty members.

Source: FSU Institutional Effectiveness Portal, 2016-17.

23

6. Instructional Demands To what extent, does the unit have difficulties meeting instructional demands? If

difficulties do exists, please explain the cause(s) and impact(s).

The M&S program has had virtually no problem in meeting instructional

demands in recent years. Our full-time faculty all cooperate and offer various

combinations of service courses and advanced classes to support the dual

missions of the program. In addition, when the need arises to offer additional

sections of fundamental courses, we have a cadre of graduates who are

capable and who have taken our courses, so are very familiar with our

teaching style and (obviously) the course content. These grads are employed

locally, often by the Florida Department of Education, and enjoy teaching as

adjuncts.

On several occasions, prior to our being at the current size of five faculty, the

program was allowed to hire a full-time adjunct to cover courses. In two cases

we hired program graduates who had chosen to remain in Tallahassee for

family reasons. On two other occasions we made outside hires; those were

less successful because of the mismatch in content understanding and

teaching style.

24

7. Distance Learning Outline the role distance learning plays in the current curricular and degree offerings. Be

specific.

a. What future plans does the unit have for utilizing distance learning technology?

b. How are online course monitored to ensure they are equivalent in rigor to

traditional courses?

As much of both our doctoral and Master’s experience involves individual

mentoring of graduate students, we do not currently plan to offer an online

degree. However, as several of our courses are commonly taken by students

in other programs which do offer online degrees, we offer the introductory

course EDF 5400 online. In the future, we may move some of the more

popular courses in the core sequence, such as EDF 5401, 5402, and 5406,

online as well. These courses would be offered with both online and face-to-

face (or mixed) options.

To move EDF 5400 online, the lectures used in the face-to-face class were

recorded using screen capture technology. The lectures were then edited into

smaller segments, and a small self-check quiz was added after each segment.

Case studies and practice problems, which had previously been done in class,

are now done through discussion groups. (Using the recorded lectures, but

doing the practice problems and case studies in the classroom allows for a

mixed or “flipped” classroom.) The homework assignments and lab activities

are identical in both classes, and the midterm and final are built to the same

specifications.

We are confident that the two courses are equivalent because the majority of

the scored material (homework, labs, and exams) is the same or comparable

(minor variations from semester-to-semester) for both classes. Furthermore,

the same instructor (Almond) has been responsible for both the face-to-face

and online versions of the class, serving as a resource when other instructors

have taught the course. Finally, the course has been offered several times

with both online and face-to-face sections in the same semester. In those

cases, the same final exam was used, with no significant difference in the

scores between the two sections.

25

8. Common Prerequisites (Undergraduate Only)

Review and report on the unit’s compliance with State-approved common prerequisites

(https://dlss.flvc.org/admin-tools/common-prerequisites-manuals).

NA. The program in M&S offers only graduate degrees, thus has no

undergraduate majors.

26

Student Experience:

9. Limited Access (Undergraduate Only)

If the degree program(s) is approved by the Board of Governors as a limited access

program, is the current implementation strategy yielding the quality of students desired?

If not, what adjustments does the unit anticipate making to improve recruitment of

students prepared to succeed? Does the degree program/major still warrant limited access

status? (Review Regulation 8.013 at

http://www.flbog.edu/documents_regulations/regulations/8_013_Limited_Access.pdf.) If

so, explain why by linking the current reasoning to the regulation language.

NA. The program in M&S offers only graduate degrees, thus has no


27

10. Advising Explain how the unit handles advising (graduate and undergraduate).

The program in M&S offers only graduate degrees, thus has no


Graduate students in the Measurement and Statistics program are selected

for admission on the basis of GRE scores and GPA, prior training and

accomplishments, personal statements, and three letters of recommendation.

In some cases students are also interviewed in person or by telephone or

skype. These admissions data are then used to assign a temporary advisor,

who is identified and assigned before the student arrives at FSU. When

students express an interest in working with a particular faculty member,

that person is assigned as advisor if at all possible. In both master’s and

doctoral degree programs, advising is done one-on-one, and every student is

expected to meet with their advisor at least once per term, regardless of their

stage of progress.

Most students seek out their temporary advisor for a meeting as soon as they

arrive on campus. If that has not happened by the time of the orientation

(which happens early every fall), the occasion is used to facilitate those

meetings. Early in their FSU careers, and particularly for students in the

coursework master’s degree programs, students may only need to meet their

advisors once or twice per term, to plan for coursework and to keep in touch

on how coursework is progressing.

In addition, the M&S colloquium series provides many ongoing opportunities

for contact and group advising. Topics include the structure of exams and

requirements, and availability of university resources (see also Question 15).

Attendance at colloquia is required and colloquium presentations are

frequently recorded for later student review.

Most doctoral students begin early to have close contact with their advisors.

Some start to work on projects almost immediately, but this depends on the

student’s background. The most difficult period of time from the faculty

perspective is when students are working most independently -- during the

prospectus and dissertation period -- because some students become isolated

(sometime because of feelings that they have not made adequate progress).

All advisors are aware of this issue and attempt to make sure students see

them as accessible and supportive as the students become more independent

researchers.

28

The doctoral supervision load is not evenly distributed among faculty

members. As students often select senior faculty to be their mentors, senior

faculty tend to end up with supervising more doctoral students than junior

faculty. Also the department policy is to protect junior faculty from

exceedingly heavy advising loads. However, because of the excellent

collaborative efforts among the faculty members, supervision inequalities

have not caused too many issues among the students and the faculty

members.

29

11. Instructional Evaluation Outline how the unit conducts and utilizes instructional evaluations—faculty, adjunct

instructors and teaching assistants.

Instructional evaluation of faculty members. Faculty are observed during

their first terms and given detailed feedback about their performance.

Evaluations of teaching effectiveness are conducted in all classes faculty

members are teaching (using the State University System Student

Assessment of Instruction (SUSSAI) forms and later the Student Perception

of Courses and Instructors (SPCI) forms, starting in 2013). New faculty

members generally have chosen to sit in on other sections of the courses they

are teaching to observe more-senior faculty members teach, which has been

quite effective. In addition, senior faculty members have often volunteered to

attend the classes of new faculty during their first terms, thus allowing for

immediate feedback and improvement.

If SUSSAI/SPCI feedback includes many “fair” or “poor” ratings for an

instructor, the chair is obliged to communicate with the instructor and

implement plans to overcome the problem. On some occasions other faculty

have been asked to observe the person’s class and provide feedback and

written reports. These are very rare cases, as most of the faculty members in

M&S are considered to be very good or excellent instructors. The table titled

“Faculty SPCI Scores for Courses Offered from Fall 2013 to Spring 2017”

provides students’ mean ratings for each course offered in the last four years

on the SPCI item “Overall Rating for Instructor”. Students were asked to rate

on a scale with 5 = excellent, 4 = above satisfactory, 3 = satisfactory, 2 =

below satisfactory, and 1 = poor. As evidenced in the table, many of the

median scores are at the highest level (4 or 5), representing "excellent" or

“above satisfactory” in instruction. The average of mean ratings across all 95

sections of M&S classes is 4.33 (SD = 0.54), with courses taught by regular

faculty members (M = 4.43, SD = 0.49) being rated slightly higher than the

courses taught by adjunct instructors (M = 4.12, SD = 0.58).

Instructional evaluation of adjunct instructors: In recent years all adjunct

instructors except one have been program graduates whose capabilities for

teaching were known before hiring. The exception is Dr. William Yeaton who

was on the faculty of Ball State University and also employed as a research

associate at the University of Michigan for many years before his retirement.

He has taught an advanced seminar course “Experimental/Quasi

Experimental Designs” for us in every Spring semester (since 2011). Review

of adjunct instruction includes analysis and discussion of teaching

performance, observations of instruction by tenure-earning or tenured

program faculty, and the like. In all cases SUSSAI/SPCI forms are reported

30

to the department chair. In recent years these ratings have not caused M&S

to terminate or not renew employment of any part-time or temporary faculty

member. As reported in the table, our adjunct instructors are overall good

instructors with average student ratings of 4.12 (SD = 0.58).

Instructional evaluation of teaching assistants: Students in M&S are not

required to teach and are not allowed to teach graduate-level courses. All but

one teaching assistant (TA) in M&S have served as graders, lab supervisors,

and have held office hours for courses under the supervision of a faculty

member. One past student did serve as instructor of record for our

undergraduate classroom assessment course (EDF 4210).

To ensure high quality of TA work, TAs have to meet the University-wide

Standards for Graduate Teaching Assistants (available at

http://pie.fsu.edu/sites/g/files/imported/storage/original/application/97344489

81fe21c6994025b4d17f2b4b.pdf). All TAs attend the PIE Teaching

Conference sponsored by the Program for Instructional Excellence (PIE)

before beginning their teaching assistant responsibilities. All TAs who are

not native speakers of English are required to take the SPEAK test

administered by The Center for Intensive English Studies (CIES) and score

45 or higher on the test (students who score 26 or higher on the speaking

portion of the IBTOEFL may be exempted from taking the SPEAK test).

The main expectations for TAs are: (1) know the subject matter, (2) respect

students and their needs, (3) use more than one method for explaining

concepts and materials, (4) provide reinforcement and supporting feedback,

and refrain from criticism and destructive comments, (5) give and evaluate

assignments fairly – based on criteria which are clear to the learners, and (6)

be sensitive to learning and other disabilities and limitations. As part of

departmental requirements, any graduate student who is employed as a

teaching assistant receives a written performance evaluation at the end of

the semester by the faculty-member who is responsible for the student’s

work. TAs who are not performing their duties well are first provided with

feedback and support, and monitored closely. If they cannot improve their

performance, they are not rehired in later semesters.

As mentioned above, over the last seven years, only one doctoral student

taught the undergraduate level Classroom Assessment course. This student

is a native speaker and thus was not required to take the SPEAK test.

Classroom Assessment has been supervised and evaluated by faculty in

Learning & Cognition. Teaching evaluation and training of graduate

instructors are described in Question 20.

http://pie.fsu.edu/sites/g/files/imported/storage/original/application/9734448981fe21c6994025b4d17f2b4b.pdf

http://pie.fsu.edu/sites/g/files/imported/storage/original/application/9734448981fe21c6994025b4d17f2b4b.pdf

31

Faculty SPCI Scores for Courses Offered from Fall 2013 to Spring 2017.

Instructor Course Title n Response

Rate (%)

Mean

Rating

Betsy Becker EDF5401 GEN LINEAR MODEL 21 80.95 4.53

Betsy Becker EDF6937 META-ANALYSIS 18 83.33 4.87

Betsy Becker EDF6937 SURVEY RESEARCH 8 62.5 5

Betsy Becker EDF7489 META-ANALYSIS 15 100 4.33

Insu Paek EDF5402 ADV TPC ANLY VARANCE 30 96.67 3.72

Insu Paek EDF6937 MULTIDIM IRT 6 100 4.5

Insu Paek EDF5432 MEASURMNT THEORY I 24 91.67 4.09


Insu Paek EDF7418 MULTILEVEL MODELING 17 82.35 4

Insu Paek EDF5434 MEASURMNTS THEORY II 10 90 4.56

Insu Paek EDF5401 GEN LINEAR MODEL 18 83.33 4.13

Insu Paek EDF6937 BAYESIAN IRT 5 100 5

Insu Paek EDF6937 LATENT VAR MODELING WITH R 6 83.33 5





Insu Paek EDF6937 ADVANCED IRT 5 100 5



Insu Paek EDF5432 MEASURMNT THEORY I 19 100 4.21

Insu Paek EDF5434 MEASURMNTS THEORY II 9 88.89 5


Qian Zhang EDF5401 GEN LINEAR MODEL 12 75 4

Qian Zhang EDF7418 MULTILEVEL MODELING 18 100 4.47

Qian Zhang EDF5401 GEN LINEAR MODEL 28 78.57 4.85

Qian Zhang EDF6937 LONGIT. DATA ANALYSIS 11 100 4.82

Qian Zhang EDF5401 GEN LINEAR MODEL 10 100 4.8

Qian Zhang EDF5401 GEN LINEAR MODEL 16 93.75 4.6

Qian Zhang EDF7418 MULTILEVEL MODELING 18 83.33 4.8

Russell Almond EDF5400 DES/INF STATSTCS APP 39 58.97 4.35

Russell Almond EDF5448 SCALE/INSTRUMENT DEV 8 100 4.5


Russell Almond EDF6937 BAYES DATA ANALYSIS 7 100 3.29


Russell Almond EDF6937 BAYES NET 8 75 3.5



Russell Almond EDF5448 SCALE/INSTRUMENT DEV 8 87.5 4.14

Russell Almond EDF5448 SCALE/INSTRUMENT DEV 19 84.21 4

32



Russell Almond EDF6937 MISSING DATA 8 75 4.2

Russell Almond EDF5400 DES/INF STATSTCS APP 29 75.86 4


Russell Almond EDF6937 ED DATA ANALYSIS 5 100 4.6

Yanyun Yang EDF5409 CAUSAL MODELING 8 100 5

Yanyun Yang EDF5406 MULTIVARIATE ANAL 32 84.38 4.81



Yanyun Yang EDF5409 CAUSAL MODELING 30 83.33 4.76



Yanyun Yang EDF6937 ADVANCED SEM 5 100 5

Yanyun Yang EDF5406 MULTIVARIATE ANAL 14 100 4.79



Yanyun Yang EDF6937 ADVANCED SEM 5 100 5




Yanyun Yang EDF5406 MULTIVARIATE ANAL 6 100 5

Yanyun Yang EDF6937 MSMT INVARIANCE 9 100 5

Salih Binici EDF5401 GEN LINEAR MODEL 13 69.23 4.56

Salih Binici EDF5435 THEORY SCALING 7 71.43 5

Salih Binici EDF5401 GEN LINEAR MODEL 22 63.64 3.71

Salih Binici EDF5402 ADV TPC ANLY VARANCE 27 44.44 3.09

Salih Binici EDF6937 SEM ADV RESERCH PROB 13 69.23 4.11

Salih Binici EDF5402 ADV TPC ANLY VARANCE 25 80 4.1

Salih Binici EDF5435 THEORY SCALING 6 83.33 5

Salih Binici EDF6937 SEM ADV RESERCH PROB 6 66.67 5

Salih Binici EDF5402 ADV TPC ANLY VARANCE 19 52.63 3.5

Salih Binici EDF5400 DES/INF STATSTCS APP 17 76.47 4.31

Salih Binici EDF6057 LARGE-SCALE ASSESS. 5 100 4.8

William Yeaton EDF6937 EXP/QUASI. DESIGN 9 88.89 4.25

William Yeaton EDF6937 EXP/QUASI. DESIGN 15 93.33 5



Christine Ouma EDF5400 DES/INF STATSTCS APP 23 65.22 4

Tracey Gunter EDF5400 DES/INF STATSTCS APP 33 66.67 3.14

Tracey Gunter EDF5400 DES/INF STATSTCS APP 25 76 2.89

Tracey Gunter EDF5400 DES/INF STATSTCS APP 10 80 3.43

Ying Zhang EDF5400 DES/INF STATSTCS APP 38 26.32 4.2

33

Ying Zhang EDF5401 GEN LINEAR MODEL 29 82.76 3.21



Ying Zhang EDF7489 META-ANAYLYSIS 18 88.89 4.25

Ying Zhang EDF5406 MULTIVARIATE ANAL 16 56.25 4.89

Ying Zhang EDF5401 GEN LINEAR MODEL 22 81.82 4

Ying Zhang EDF5400 DES/INF STATSTCS APP 22 68.18 4

Ying Zhang EDF5406 MULTIVARIATE ANAL 21 80.95 4.38




Ying Zhang EDF5402 ADV TPC ANLY VARANCE 34 76.47 3.88

34

12. Graduate Placement Insert Table 4.

Table 4 consists of data gathered as part of the Senior Exit Survey.

a. Discuss placement rates and quality of placements for graduates of the degrees in

the unit (e.g., job placement rates, graduate school placement rates, types of jobs,

caliber of graduate schools). Please be as specific as possible.

b. If applicable, provide and analyze pass rates of important external

examinations/licensures for the last five years. Include the total number of

students attempting the exam(s) and the total number who pass. Comment on the

passage rate of FSU students versus national norms. What action is the unit

undertaking to improve student performance on such national indicators?

Note Table 4 is not available for graduate programs and Question 12b is not

applicable to the M&S program.

Students in M&S majors are being prepared for academic career tracks, and

for professional positions in industry, in national, state, and local research

and education offices, and in private evaluation and consulting firms.

In terms of academic placements, doctoral graduates have taken positions in

both teaching-oriented universities and Tier I and II research universities. Of

23 recent doctoral graduates, 13 have taken teaching or research positions in

universities, including Rutgers University, Eskisehir Osmangazi University

(Turkey), University of Arkansas, Arizona State University, Texas A&M

University, Karadeniz Technical University (Turkey), Yildiz Technical

University (Turkey), and Emporia State University.

In addition to academic postings, the M&S program has placed graduates in

the testing industry, as well as in Florida Department of Education’s

Psychometrics Office. Of 23 doctoral graduates, 9 have worked as

psychometricians in testing companies and certification boards (e.g., AIR,

Amplify Education Inc., and American Nurses Association), Florida

Department of Education, and Korea Institute for Curriculum and

Instruction, and 1 has worked in the Office of Faculty Development at FSU.

Placements of master’s graduates have not been tracked consistently,

however, we know that many continue on to doctoral study at FSU or

elsewhere, and others have taken positions in professional postings (e.g., data

analyst in the office of institutional research in universities; program

specialist at FLDOE).

35

13. Enrollment Trends Insert Table 5.

Table 5 presents enrollment trends by degree program for the past seven years along

with data related to student diversity.

a. Comment on the enrollment and retention levels and the number of degrees

granted for the past seven years. Analysis should include an explanation of trends

by degree program and whether the unit anticipates changes in the pattern over

the next five years. Include analysis of racial and gender diversity.

b. What steps is the unit taking to increase diversity among its student population?

a. Table 5 shows data on enrollees by year. The enrollment in the doctoral

program was 27 at the end of the last QER period. The enrollments were

constant with 26 students from Fall 2010 to Fall 2012, increased to 31 in Fall

2013, and then gradually declined to 20 in 2016. Two reasons may explain

the up and down in enrollments across years. On one hand, the number of

newly enrolled doctoral students was relatively stable across years (see Table

8 in Appendices). On the other hand, the number of doctoral students

finishing their degrees differed over the years, with 0 in 2012-2013 and 4 in

2014-2015 and 2015-2016. The enrollment trends in the Master’s program

showed a similar pattern.

In the last few years, M&S has definitely had difficulties in student

recruitment. We use most of the traditional means to recruit students.

Referrals from graduates are common. Faculty receive and answer much

direct email from potential students, and web sites attract attention, though

we believe that the web sites could be more detailed in what they say about

faculty and program activities. We have also created a Facebook page to have

more flexibility in this regard. Our program is listed in the American

Psychological Association’s quantitative psychology program list.

With all of these efforts, we were in the past able to attract a sufficient

number of applicants. However, many recently admitted students decided not

to enroll due to a lack of financial support. We expect the same issue will

continue if there is no additional support. For example, of 24 doctoral

program applicants in 2013-2014, we admitted 15, but only 3 enrolled. At the

end of 2016, the number of enrolled doctoral students was 20, with the

majority being doctoral candidates. The number of enrolled doctoral students

will decline in the upcoming years once these candidates finish their degrees,

unless we raise our enrollment rate. The M&S program used to be attractive

to government-supported applicants from Turkey. Unfortunately for a variety

of reasons, largely political, the number of such applications has also declined

in the past two years.

36

Table 5 provides the gender and race/ethnicity breakdown for both the

master's and doctoral programs. Roughly half of M&S students are female.

The programs are both dominated by international and minority students.

This trend has been relatively stable for many years.

b. Because our program is gender balanced with slightly more women than

men, our recruitment has been gender neutral. The M&S program has been

paying special attention to the recruitment of minority students in the

student application review and admission evaluations. The M&S program

will continue to monitor the recruitment, retainment, and graduation of

minority students. To further diversify the student pool, the M&S program

will continue to encourage and monitor the recruitment of underrepresented

U.S. students including ethnic minority students in the U.S. through

advertisement of the M&S program and meetings to promote dialogue

between M&S faculty and potential applicants (e.g., advertisement at local

and professional conferences and guest speaker invitation talks).

Additionally, the M&S program will continue to provide supportive financial

resources to promote diversity through actively maximizing minority

scholarship programs offered by the college and the university.



Table 5: Graduate Program Enrollment Trends

Graduate Headcount Enrollment by Degree Level Sought

Gender Race/Ethnicity M D M D M D M D M D M D M D

American Indian/Native Alaskan 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Asian 1 0 0 1 0 1 0 1 0 1 0 1 0 1

Black 0 1 0 1 0 1 1 1 1 1 1 1 0 1

Hispanic 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Native Hawaiian or Pacific Islander 0 0 0 0 0 0 0 0 0 0 0 0 0 0

White 0 1 0 1 1 1 1 1 0 0 0 0 0 0

Non‐Resident Alien 2 14 4 10 5 10 5 11 5 11 4 10 2 8

Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Not Reported 0 0 0 0 0 0 0 0 0 0 0 0 0 0

3 16 4 13 6 13 7 14 6 13 5 12 2 10


Asian 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Black 0 1 0 1 0 1 0 1 0 1 1 0 1 0

Hispanic 1 0 1 0 0 0 0 0 0 0 0 0 0 0


White 2 4 2 5 2 4 1 4 1 3 0 1 0 0


Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Not Reported 0 0 0 0 0 0 0 0 0 0 0 0 0 0

3 10 4 13 4 13 1 17 2 16 3 13 2 10

6 26 8 26 10 26 8 31 8 29 8 25 4 20

M = Masters; D = Doctorate

Source: Fall Preliminary Student Instruction Files (SIFP)

Fall Fall

2010 2011 2015 2016

Fall Fall Fall Fall Fall

Female

Male

2012 2013 2014

33 24

Degree Level Total

Grand Total

Female Total

Male Total

32 34 36 39 37

68

108 8 8

4

26

26

26 3129

25

20

0

5

10

15

20

25

30

35

40

45

2010 2011 2012 2013 2014 2015 2016

Master's Doctorate



Table 5: Graduate Program Degrees Awarded Trends

Graduate Degrees Awarded by Level

Gender Race/Ethnicity M D M D M D M D M D M* D

American Indian/Native Alaskan 0 0 0 0 0 0 0 0 0 0 0 0

Asian 1 0 1 1 0 0 0 0 0 0 0 0

Black 0 0 0 0 0 0 0 0 0 0 0 1

Hispanic 0 0 0 0 0 0 0 0 0 0 0 0

Native Hawaiian or Pacific Islander 0 0 0 0 0 0 0 0 0 0 0 0

White 0 0 0 0 0 0 1 0 0 1 0 0

Non‐Resident Alien 3 2 1 1 2 0 1 1 4 3 3 3

Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0

Not Reported 0 0 0 0 0 0 0 0 0 0 0 0

4 2 2 2 2 0 2 1 4 4 3 4


Asian/Pacific Islander 0 0 0 0 0 0 0 0 0 0 0 0

Black 0 0 0 0 0 0 0 0 0 0 0 0

Hispanic 0 0 1 0 0 0 0 0 0 0 0 0


White 0 0 1 0 1 0 0 1 0 0 0 2


Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0

Not Reported 0 0 0 0 0 0 0 0 0 0 1 0

3 0 3 1 2 0 0 2 0 0 3 4

7 2 5 3 4 0 2 3 4 4 6 8


* Master's includes one conferred Specialist’s degree

Source: Student Instruction Files ‐ Degrees (SIFD)

5

2011‐12 2012‐13 2013‐14 2014‐15 2015‐16

Grand Total

2010‐11

9 8 14

Male Total

Degree Level Total

Female

Female Total

Male

8 4

7

54

2

4

6

2

3

03

4

8

0

2

4

6

8

10

12

14

16

2010‐11 2011‐12 2012‐13 2013‐14 2014‐15 2015‐16

Master's Doctorate

37

14. Time to Degree What steps is the unit undertaking to ensure that students complete degrees in a timely

manner?

We anticipate non-thesis track students at the master's level will complete

degrees in 2 years or less, and thesis-track masters’ students in 2-3 years. At

the doctoral level, students are expected to complete the degree within 4 to 6

years, depending on whether they obtained a master’s degree in a relevant

field, and whether they continue the doctoral degree directly from our own

master’s program. The time frame is reasonable considering the need for

most students to take extensive coursework from the Department of

Statistics.

Faculty members help ensure that students complete their degrees in a

timely manner by closely monitoring their progress, and most critically,

getting students involved in research activities earlier in their student

careers. This is facilitated by having students join active research groups and

requiring attendance at the bi-weekly colloquia, which focus on ongoing

research, preparation for conference presentations, information on job

seeking, and preparation of vitas.

The table entitled “Ph.D. Time to Degree” in Appendices summarizes the

number of years students have taken to complete the doctoral degree during

2010-2017 academic years. Among 23 doctoral graduates, 15 took 6 years or

less, 4 took 7 years, 2 took 8 years, and 2 took 10 years. Those who have

taken 7 or more years were either students who were already working as full-

time employees or students who were dual-enrolled in the Master's degree

program in the Department of Statistics, which requires more credit hours to

finish the coursework.

38

15. Graduate Student Preparation (Graduate Only)

Explain how the unit addresses the following within its graduate degree programs:

a. Professional development and exploration of research integrity.

b. If appropriate, career exploration/preparation for employment outside of

academia.

c. For doctoral programs, fulfillment of the scholarly engagement requirement.

a, b. Preparation for all professional roles is done by way of direct mentoring

and in some cases by special instruction. Mentoring for teaching occurs as

students work as teaching assistants (TAs). All TAs in M&S are responsible

for scoring, lab supervision, and holding office hours. They are often given the

chance to do direct instruction, for instance, by teaching a specific lecture

topic or running an exam review session. Also as discussed in Question 11,

TAs must attend PIE sessions and take the SPEAK language competency

test sponsored by the university.

Mentoring for research occurs as students work with faculty on research

projects of common interest, and also via some of the milestone

accomplishments required for the degrees (qualifying exam and preliminary

exam). We have discussed how advisors/mentors supervise doctoral students

in Question 10. Some faculty work with their advisees in a one-on-one style.

Several faculty members have ongoing research groups to support student

work. For example, Becker's Synthesis Research Group (SynRG) has existed

for over 20 years, since well before she joined the FSU faculty. Biweekly talks

by both students and other members allow for feedback on developing

interests as well as more formal practices for conference presentations.

SynRG regularly hosts international visiting scholars (e.g., from Germany,

South Korea, and Spain), and provides a forum for interaction and

collaboration among students, graduates, and others interested in meta-

analysis from across campus.

In addition to working directly with faculty on research projects, students in

M&S have a colloquium series in which they can present their work and

react to the presentations of others. This has been a forum for presentation of

faculty interests, for development of speaking and writing skills, and for

learning about human subjects and research ethics issues such as plagiarism

and authorship issues. Some sessions cover potential employment

opportunities for graduates, including non-academic options. Special sessions

are scheduled for students to get practice on their presentations for American

Educational Research Association, AERA, FERA, etc. Several research

groups, including the R User Group, allow students to present on ongoing

research or other topics to practice professional presentation skills.

39

Also, the M&S goals for the Southern Association of Colleges and Schools

(SACS; see Table 3) involve components of graduate student preparation.

Specifically, one goal is to have 100% of students beyond the first year joining

professional organizations, and for students in the second year of the doctoral

program to be involved in conference presentations. Involvement can include

being part of the submitted proposal, or being an author and presenter or a

co-author on an accepted presentation.

c. Starting from Fall 2016, M&S has implemented a new set of Scholarly

Engagement requirements to encourage doctoral students to be more engaged

in professional development and research. Specifically, doctoral students in

Measurement and Statistics can meet the Scholarly Engagement

requirement by fulfilling either items 1 and 2, OR items 1 and 3, shown

below.

(1) Doctoral students should attend the bi-weekly M&S colloquia meeting

unless an excused absence is provided. Excused absences include

documented illness, deaths in the immediate family and other documented

crises, call to active military duty or jury duty, religious holy days, and

official University activities.

(2) Doctoral students in the first two years of study should attend a

professional conference (e.g., American Educational Research Association,

American Psychological Association, American Statistical Association, or

Florida Educational Research Association) that is related to their program

of study. Doctoral students in their third year of study and beyond should

give one or more presentation(s) related to their program of study, such as

being an (co)-author in a professional conference.

(3) Doctoral students should be involved in preparing, submitting, or

publishing research paper(s) to scholarly outlets (e.g., journals, books) as

authors or coauthors.

For the 2016-2017 academic year, the average attendance rate for the M&S

colloquia was 48%. Excluding students who are not active due to their full

time employment or other reasons, the average attendance was 55%.

For the 2016-2017 academic year, the percentage of doctoral students who

fulfilled requirement (2) was 72%. The percentage of doctoral students who

fulfilled requirement (3) was 50%. For the doctoral students who showed

more than 50% attendance in spring 2017, the percentage of those who

fulfilled either requirement (2) or (3), or both (2) and (3) was 88%.

40

Faculty:

16. Scholarly/Creative Overview Provide an overview of the scholarly/creative agenda for the unit faculty. Highlight

particular strengths.

The five associated faculty in the Measurement & Statistics program have

research interests that cover many areas in the field that together answer a

variety of important methodological and empirical questions in educational

and psychological studies.

Almond is interested in answering the question of how to gather, track and

monitor evidence of student growth using both traditional sources (e.g.,

assessments and homework) and non-traditional sources (e.g., simulations

and games), especially in situations where multiple dimensions of student

proficiency are considered. The book Bayesian Networks in Educational Assessment written by Almond and colleagues was published in 2015. He is

collaborating with Shute and Ke from the Instructional Systems and

Learning Technologies program on Game-based Assessment and Support of STEM-related Competencies (funded by the National Science Foundation, for

$1,066,257). He has two external grant proposals that are under review, one

on Mathematical Learning and the other on Teacher Decision Making.

Becker is interested in the area of meta-analysis and also in psychometric

issues in education. Her current research involves methods for synthesizing

correlation matrices and regression slopes. She has also been involved in

synthesis projects regarding teacher knowledge and teacher qualifications,

diabetes management, and cancer outcomes. In the past five years, Becker

has been funded by the National Science Foundation ($475,901) on the

project Collaborative Research: Partial and Multilevel Effect Sizes in Meta-Analysis. She is a co-investigator on a pending multi-million dollar center

grant in the area of reading difficulties.

Paek is interested in psychometric issues in educational/psychological

testing, test construction, and large scale assessment. More specifically, he is

interested in studying item response theory modeling and its extensions,

including differential item functioning (item bias) techniques, and test score

equating (IRT or observed score approaches for vertical scaling and

horizontal equating).

Becker and Paek are in charge of mentoring interns sent from the

Measurement and Statistics program to work with the Florida Department of

Education.

41

Yang’s major research interests include reliability estimation methods, factor

analysis, and structural equation modeling, particularly related to categorical

data. She is also interested in applications of advanced statistical procedures

to subject areas such as mental health, educational psychology, and sport

psychology. She served as a key personnel member on the project An Alternate Statewide Assessment Strategy That Uses Test Results to Support Learning and Includes Measures of Problem Solving funded by the U.S.

Institute of Education Sciences (for $2,000,000).

Zhang’s research interests focus on mediation and moderation analyses,

multilevel modeling, longitudinal data analysis, and missing data problems.

Her current projects include developing and estimating two-level and three-

level longitudinal mediation models, handling missing data in moderation

analysis, and latent moderation modeling. Substantively, she is interested in

applying statistical methods in developmental, educational, and psychological

research. She and her colleagues have submitted two grants, one on Life Review with Nursing Home Residents at the End of Life, the other on

Random Coefficient Meta-Analytic Structural Equation Modeling. In addition, associated faculty bring expertise and interests in large-scale

assessment, and experimental design. Some of them have overlapping

research areas that promote collaborations and mentoring of graduate

students.

42

17. Scholarly/Creative Productivity Insert Table 6.

Table 6 presents various measures associated with faculty creative/scholarly

productivity.

Analyze the faculty productivity information presented in Table 6. Include, as

appropriate, comments related to grants (PI and co-PI), publications and citations.

If additional measures of faculty productivity are appropriate for the unit,

please provide a brief explanation of the significance of the measure(s) and

comment on related faculty productivity.

As can be noticed in our CVs, our five faculty members in the Measurement

& Statistics program have published over 70 countable publications

(including journal articles, books and book chapters, monographs,

proceedings, and reports) over the past five years. The numbers of citations

provided in Table 6 are for publications between 2011 and 2015 only, and

only in a limited number of outlets. If we look at citations for all published

articles, the numbers of citations for most faculty members are much higher.

For example, since 2012, Almond, Becker, Paek, and Yang had 1687, 5713,

200, and 826 citations based on Google Scholar and Scopus Preview.

Our five associated faculty publish in the leading journals of our field, such as

Journal of Educational and Behavioral Statistics, Psychological Methods,

Multivariate Behavioral Research, Psychometrika, and Structural Equation Modeling. Some of the journals where our faculty have published were not

included in the provided List of Included Journals (e.g., Technology, Instruction, Cognition and Learning, KEDI Journal of Educational Policy,

Journal of Modern Applied Statistical Methods, Journal of Clinical Epidemiology, Research in the Sociology of Organizations, Journal of Applied Measurement, The International Journal of Educational and Psychological Assessment).

Faculty in M&S have also made over 100 presentations in national and

international conferences during the QER evaluation timeframe. Senior

faculty have given invited or keynote presentations all over the world. (See

the QER vitae of faculty which document these accomplishments.) Also, our

faculty members are “grant active”. Over the past five years the senior

faculty members were either principal investigators (PIs) or co-PIs on state or

federal grants. Over $1.5 million in federal grants have been garnered over

the past 5 years, as well as an additional $711,851 in state and other grants.

Junior faculty member have served as named personnel, and are involved in

pending grant applications as collaborators.

43

Faculty members have served as officers in state, national, and international

professional societies, and/or have held editorial responsibilities. Almond is

the associate editor of Behaviormetrika. Becker is currently on the editorial

board for Research Synthesis Methods and Journal of Research in Rural Education. Paek is on the editorial board of Asia Pacific Education Review. Yang is the associate editor of Journal of Psychoeducational Assessment and

Behavior Research Methods. All five faculty members are active as editorial

referees for various methodological and applied journals.


Educational Psychology and Learning Systems | Measurement and Statistics Program

Table 6: Faculty Productivity

Faculty MemberFall 2015

RankDegree Date

Articles

(2012‐15)

Citations

(2011‐15)

Books

(2006‐15)

Awards

(no limit)

PI & Co‐PI

Proposals

(2012‐16)

PI & Co‐PI

Expenditures

(2012‐16)

Courses

Taught

(2015‐16)

Enrolled

Students

(2015‐16)

Almond, R. Associate 1990 5 7 1 0 5 $48,799 16 71

Becker, B. Professor 1985 7 28 1 2 10 $1,047,316 16 47

Paek, I. Assistant 2002 12 27 0 0 4 $0 17 66

Yang, Y. Associate 2007 12 131 0 0 4 $0 11 87

List of journals: http://ir.fsu.edu/qer/2017/references/AA_2015_Journals_List.pdf

List of awards: http://ir.fsu.edu/qer/2017/references/AA_2015_Awards_List.pdf

Books, articles, and

citations: http://ir.fsu.edu/qer/2017/references/AA_2015_Criteria_for_Books_and_Citations.pdf

44

18. Faculty Workload How does the unit monitor and adjust the distribution of workload among faculty

including committee work and graduate student committees? Please include a link to the

related policy statement or section of the bylaws.

With only five full-time faculty in the program, most decisions are done by

consensus either in face-to-face meetings or through email. No bylaws govern

this process.

Teaching: Our program has a number of core service courses that must be

offered each semester as well as a number of courses required for the

program which must be offered at least once every two years. In addition it is

important to occasionally offer seminar courses which expose students to

breaking research frontiers. Full-time faculty members have a load of 4

courses: most do two service courses, one required course, and one seminar

course per year. Becker, as chair, has an administrative reduction each term

and has also had grant-funded buyouts; adjuncts are used to teach

replacement courses.

Advising: Primary advisors are assigned when students enroll in the

master’s or doctoral program. The choice is made to both a) match students'

stated research interests with faculty research interests, and b) balance the

advising roles. Students may change advisors after their first year but

historically most have continued with their assigned advisors. Students

generally select the other members of their committees themselves, often

based on their research interests, with the faculty member refusing if they

cannot make the time commitment. Untenured faculty are protected from

serving as the outside representative on a committee.

Service (within the program): Many program activities (e.g., admissions,

writing and scoring the major exams, course assignment) are shared among

all program faculty. The workload is again balanced by consensus. Other

assignments are made by consensus. Yang serves as program coordinator,

Paek coordinates the certificate program and Almond runs the colloquium

and computer lab. (Becker currently serves as department chair and Zhang

has not yet been given a permanent program assignment).

Service (Department, College and University levels): Members of department

standing committees are chosen at the Spring faculty meeting. Usually, the

positions are filled by asking for volunteers, with the chair facilitating the

process to ensure a reasonable distribution of service load. In particular, the

bylaws state (VI.B.1) “As a general rule, during the committee term, no EPLS

faculty member should serve on more than two EPLS standing committees.”

45

For many of the committees the bylaws require one representative to be

chosen from each of the three program areas: Psychological and Counseling

Services, Instructional Systems and Learning Technologies, and Educational

Psychology. For the purpose of committee assignments, Measurement and

Statistics falls under Educational Psychology, together with the Learning

and Cognition and Sports Psychology programs.

Memberships on college committees are also chosen at the spring department

meeting, and nominations for faculty senate representation are made at that

meeting as well. The general collegial attitude of the department ensures

that the service load is fairly shared. Tenure-earning faculty are protected

from high workload assignments, especially during their first couple of years.

46

19. Faculty Retention What issues, if any, negatively impact faculty retention efforts?

Faculty retention in Measurement and Statistics is not a problem. The work environment

is very positive and aside from relatively low faculty salaries (a university-wide issue)

resources are very good.

a. Is there a strategic plan for filling vacancies that may occur within the next five

years, including expected retirements?

b. What steps is the unit taking to increase the diversity of its applicant pools for

faculty positions?

In the next five years senior faculty member Becker will likely retire. This

will mean one position with a focus on statistics (rather than measurement)

will open. It is highly likely that the position will be returned at the assistant

rank, thus the job will be to recruit a new scholar. Discussion and

brainstorming have already begun on the topical areas that would add

strength to the program once Becker is no longer a member of the group.

When faculty vacancies arise the search process includes a variety of

mechanisms for recruiting a diverse set of high quality applicants. First, the

usual advertising outlets are used, including the Chronicle of Higher Education, and the American Educational Research Association job postings.

Second, and most importantly, networks of professional contacts are tapped –

this includes by way of various quantitatively oriented listservs (Division D

of AERA, Division 5 of APA, Educational Statisticians Special Interest

Group, Society of Multivariate and Experimental Psychology listserv), and

via personal contacts of the faculty in M&S.

Special requests are made mentioning that qualified minority hires are of

great interest. However, as in many fields in higher education, the

quantitative field is no exception to the dearth of minority professionals. In

fact it is lacking more generally in professionals who are native U. S.

citizens, as can be seen in the composition of the student applicant and

student enrollee populations.

47

Resources:

20. Teaching Assistants Insert Table 7.

Table 7 presents student credit hour production by instructor type.

a. Comment on the proportion of course offerings taught by graduate assistants.

b. How are graduate teaching instructors selected, trained, mentored and monitored?

(Refer to University-wide Standards for Graduate Teaching Assistants at Florida

State University, located at

http://www.pie.fsu.edu/content/download/212425/1820209/University%20Wide

%20Standards%20for%20Graduate%20Teaching%20Assistants%20FINAL%201

20314.pdf

a. Over the last seven years, only one Measurement & Statistics doctoral

student has taught undergraduates, in the EDF4430 Classroom Assessment course. This student was a native speaker and thus was not required to take

the SPEAK test. Classroom Assessment has been supervised and evaluated

by the faculty in Learning & Cognition.

b. As part of departmental requirements, any graduate student who is

employed as a teaching assistant receives a written performance evaluation

at the end of the semester by the faculty member who is responsible for the

student’s work. All graduate students assigned as instructors of record are

required to have either a master’s in the same discipline or have

satisfactorily completed at least 18 semester based graduate credit hours in

the same discipline as that instruction.

To become an instructor of record for an undergraduate course, doctoral

students must first attend the University-sponsored Program in Instructional

Excellence (PIE) two-day conference where they learn about FSU's Sexual

Harassment Policy and Academic Honor Policy, the Federal Educational

Rights and Privacy Act (FERPA), Americans with Disabilities Act (ADA),

Grading Policies, Textbook Adoption Procedure Policy, Syllabus Policy, Class

Attendance Policy, Final Exam Policy, Copyright Law Regulations (Copyright

Revision Act of 1976 fair use) and Course Evaluation Policy. They also learn

about the use of Canvas for instruction, time management for Teaching

Assistants, interacting professionally with one's students, dealing with

distressed students, and diversity in the classroom, grading and assessment.

Once doctoral students become instructors of record, they are observed once a

semester by a faculty member. Faculty members observe a class session and

provide both written and verbal feedback. Additionally, undergraduate

students complete course evaluations for each graduate-student instructor. If

http://www.pie.fsu.edu/content/download/212425/1820209/University%20Wide%20Standards%20for%20Graduate%20Teaching%20Assistants%20FINAL%20120314.pdf



48

a graduate student receives overall low course ratings, the graduate student

and faculty advisor develop a plan of improvement. If the student receives

future low ratings, they may not teach that course again.

Source: Instructional Research Data Files “Other” includes adjunct faculty, post docs, staff, and overload appointments. Fall 2016 data is preliminary. The data represented in Table 7 reflect the credit hours generated by instructor type for courses classified with CIP code 130603.


Educational Psychology and Learning Systems | Measurement and Statistics Program

Table 7: Credit Hours by Instructor Type

Instructor Type Fall 2012 Fall 2013 Fall 2014 Fall 2015 Fall 2016

Faculty 1,255 1,338 833 843 897

Undergraduate SCH 0 0 0 0 0

Graduate SCH 1,255 1,338 833 843 897

Graduate Assistants 252 258 264 240 90


Graduate SCH 0 0 0 0 0

Other 132 133 191 176 92


Graduate SCH 132 133 191 176 92

Total 1,639 1,729 1,287 1,259 1,079


Graduate SCH 1,387 1,471 1,023 1,019 989

1,2551,338

833 843897

252 258 264 240

90

132 133191 176

920

200

400

600

800

1,000

1,200

1,400

1,600

Fall 2012 Fall 2013 Fall 2014 Fall 2015 Fall 2016

Total Student Credit Hours Generated by Instructor Type

Faculty Graduate Assistants Other

49

21. Adequacy of Resources Discuss the adequacy of the current resources and future needs related to:

a. Facilities

b. Technology (both locally controlled and centrally provided)

c. Staff support

Facilities. The M&S program enjoys a comfortable space, with all faculty

offices in close proximity except that of the department chair Becker. Faculty

have access to library services and technology services which support

instruction and learning, as well as research. A teaching assistant (TA) room

is specifically allocated to students in M&S who are hired as TAs for

statistics and measurement courses. A few cubicle spaces are also available

for students who are employed in certain teaching and research positions,

although with roughly 350 enrolled graduate students across the full

department, the limited set of cubicles and one TA office are hardly sufficient.

A few students have other workspaces available as part of research projects,

or via their employment on grants or other positions. The College’s Learning

Resource Center (LRC) provides study rooms and carrels for individual work,

as well as several large computer laboratories for instruction.

Technology. Information technology is provided by the department (faculty

office machines or laptops and other peripherals, and associated software) as

well as by the CoE Office of Information Technology (OIIT) and FSU IT

Services. Students and faculty use the information technology providers

intensively. The quality of information technology support the College’s IT

staff provides is outstanding. Dina Vyortkina, who oversees the operations, is

efficient and knowledgeable. It seems she and her staff always address

faculty needs promptly. LRC staff members solve local problems (e.g., in

instructional labs) efficiently and professionally.

The LRC labs provide physical facilities and software on machines in their

labs, and also the CoE also offers several statistics programs (e.g., SPSS,

LISREL, Mplus) through a virtual lab (VL) allowing students access to these

programs 24 hours a day. This is especially important for support of the

courses in statistics, which serve the entire college (and beyond). The only

complaint about the VL is that it can be slow when it is in heavy use. The lab

staff service and install statistical software as needed, and work well with

M&S faculty to be sure needs are met.

One key concern in this arena is the lack of funding for maintenance of

software licenses. FSU's Technology Fees are advertised as being available

for innovation, which often involves initial purchases of licenses for new

software for statistical analysis or other research activities. However, many

50

such licenses require annual renewal, thus the costs of future use of such

purchases falls to the department or CoE whose funds are already limited.

The department maintains two extra computers for student projects in a

special room with a separately keyed lock. The machines have many

statistical packages used in classes and student research (SPSS, SAS, R,

LISREL, Mplus, etc). Students can use these projects for large data analyses

and simulation studies that cannot be easily done on the shared LRC

computers. The locked door, and limited access to these machines, also offers

students a place to work with secure data sets. Finally, the room also serves

as an office for graders in some of the larger courses. These graders often

hold office hours for students and can use the computers to demonstrate

software being used for various classes.

A specific resource available to the members of the M&S program is a Linux

server known as pluto (located at https://pluto.coe.fsu.edu), obtained with

start-up funds upon the hiring of Almond in 2010. He maintains the licenses

for the server, and supervises users. This machine is used as a source code

control server and file sharing hub for several research and student projects.

It also provides web server resources for projects needing this as well as

serving as a computational server for large simulation studies. This machine

is now 7 years old, and in spite of some updates in terms of backup

processing, is growing near to replacement date. There is some question

about whether funds will be available for this costly but valuable machine.

In terms of other infrastructure, FSU libraries offer on-line access to most of

the important journals and books world-wide, with few interruptions. These

services help maintain high quality instruction as well as keep researchers

updated with recent software developments and publications.

Staff support. Departmental staffing and administrative structures appear

sufficient to support program functioning. The department has four staff

positions: an administrative specialist who serves as overall department

manager and who oversees three other staff members, one per program area.

The three additional staff members handle admissions for their assigned

program areas plus additional department-wide duties. Thus, for example,

one staff member handles reporting of required textbooks and admissions for

Instructional Systems & Learning Technology and Measurement & Statistics

programs; one staffer handles syllabus collections and admissions for the

Psychological and Counseling Services and Sport Psychology programs;

another handles course scheduling, course enrollments into sections of

directed independent studies, special sections, and travel expenditures, and

admissions for the Learning and Cognition program. The administrative

specialist handles appointments and tuition waivers as well as purchasing,

51

and other fiscal functions (in coordination with the department chair). This

exact arrangement has been in place only about four months but appears to

be working well to date.

At the College level, the Office of Academic Services and Intern Support

(OASIS) provides excellent support staff for graduate student monitoring and

paperwork functions. The college’s Office of Research has staff who help with

grant applications and post-award budgets.

University-level staffing does not appear to have much impact on program

matters in the department and M&S program. In cases where we have

needed to use university level offices (e.g., contact with the Dean of Faculties

for academic grievances, academic honor policy violations, etc.), the staff and

personnel have been helpful and professional in their interactions with the

department.

52

22. University Libraries Comment on the adequacy of the University Libraries services and collections as related

to the unit’s curricular programs and research.

FSU University libraries services are excellent. The FSU Libraries include 8

libraries on campus. Strozier Library, FSU’s largest library, is open 134

hours each week, providing around-the-clock research assistance and other

services like free academic tutoring and a robust range of academic support

throughout the day and late into the night. Students and faculty have a

choice of learning spaces, from the Scholars Commons’ quiet Reading Room

to the 24-hour coffee shop to the buzz of the Undergraduate Commons.

Library faculty offer classes and consultations to teach critical research and

thinking skills reaching over 27,000 participants. Seven other campus

libraries offer a many of the same services and resources, customized to

complement the disciplines they serve. For distant learners and other off-

campus library users, online research services are available, and the library

staff offers outreach to residence halls and buildings across campus.

Several library resources on campus allow on-line retrieval of information

which is current and updated. FSU libraries are connected to the majority of

the databases required for our students and faculty members. Databases

include ERIC, Education Source, Education Index Retrospective & Education

Full Text, Educators Reference Complete, and Web of Science. This set of

resources is critical to the work of Becker, whose focus on the synthesis of

research studies requires access to a wide range of research articles and

dissertations.

When resources are missing, the inter-library system and the statewide

UBorrow system secure the designated resource. The Library Express

Delivery Service delivers books and articles to faculty, post-docs, graduate,

teaching and research assistants on a daily basis. FSU Libraries also make

as many FSU publications open access as possible. They work for faculty to

upload final pre-publication versions of all manuscripts, allowing for access to

those around the world with limited library facilities. They have staff trained

in copyright law to help faculty and students navigate publication

agreements. Our faculty are beginning to make use of this service to broaden

the audiences for our work.

University Libraries offer a research consultation service for students and

faculty. Consultations are usually scheduled with a librarian or a team of

librarians to discuss some aspects of the research or publication

process. Consultations may cover many topics, including developing a search

strategy, beginning a literature review, managing reference and citations,

locating and using data sources, determining impact of research, creating a

53

data management plan, and making research open access. Our dedicated

education librarian Gloria Colvin who recently retired was top notch; she

closely monitored resources and communicated about reference issues to the

faculty in educational psychology including our program. Also having

colleague Roehrig as a chair of the FSU Library Committee has assured that

our concerns are raised and acted on.

In addition to departmental allocations, the library provides supplemental

funds to support individual faculty research in the form of: (1) New Faculty

Grants: All incoming full-time faculty receive $1,000 toward the one-time

purchase of scholarly materials to support their area of research. (2) Robert

Bradley Research Grants: On an annual basis the library allocates funds to

support competitive proposals for faculty to request research materials that

are more expensive than other funding sources can consider. Typically, at

least $50,000 is allocated. The program is administered through the Faculty

Senate Library Committee.

54

Overall Analysis:

23. Five-Year Outlook Looking to the next five years, outline opportunities that exist for the unit to enhance its

national reputation, threats or impediments that prevent taking advantage of these

opportunities, and the unit’s strategic plan for addressing the outlined threats and

impediments.

The Measurement & Statistics program is currently well placed to continue

to thrive and be seen as a high quality, small program in the areas of

educational and psychological measurement and statistics. Faculty have

interests that cover many of the field's critical areas of inquiry, including

longitudinal/growth studies and the process of mediation, item response

theory (including multidimensional IRT), Bayesian methods for

measurement, structural equation modeling, the synthesis of research (meta-

analysis), and machine learning. Associated faculty bring expertise and

interests in large-scale assessment and experimental design. Overlaps in

interests among faculty promote collaborative work and support of graduate

students.

Our main limitation and hindrance to further excellence is the lack of

funding for graduate student stipends, and lack of teaching opportunities for

our doctoral students. We regularly lose promising applicants to other

universities who offer them more funding than we can offer. Because our

program does not offer undergraduate classes, our students have no

opportunities to serve as instructor of record and receive associated graduate

assistantships. Our students do serve as scorers and teaching assistants for

our own graduate classes, but we generally have only one open position to use

for recruitment. Also many of our incoming students do not have the required

background and prior experience to step into a job as TA of our advanced

classes. Dr. Becker has two additional assistants supported by the Dean's

office for her role as department chair, but these will disappear once she

steps down. We have been urged to increase our enrollments, but without

funding mechanisms it is unlikely that we will see notable increases in the

numbers of students at the doctoral level.

55

Appendices:

Insert the following Tables:

Table 8 – Graduate Application/Enrollment Funnel

Table 9 – Graduate Program Enrollment Trends

Complete the following tables:

Ph.D. Time to Degree

Graduate Student Support Sources


Educational Psychology & Learning Systems | Measurement & Statistics

Table 8: Graduate Application / Enrollment Funnel ‐ Masters Degree

Gender Race/Ethnicity Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Nonresident Alien 6 5 1 5 3 1 3 2 1 8 2 1 3 1 0

Hispanic/Latino 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

American Indian/Alaska Native 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Asian 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Black 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0

Native Hawaiian/Other Pacific Islander 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

White 1 0 0 0 0 0 1 1 0 0 0 0 1 1 0

Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Race and Ethnicity Unknown 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0

7 5 1 6 3 1 4 3 1 9 2 1 4 2 0


Hispanic/Latino 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


Asian 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0

Black 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0


White 1 1 0 0 0 0 1 1 0 2 1 0 1 1 0

Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


3 1 0 3 0 0 3 3 1 12 5 1 2 1 010 6 1 9 3 1 7 6 2 21 7 2 6 3 0

Source: Admissions Files

2015‐16 2016‐17

Male

Male Total

2012‐13 2013‐14 2014‐15

Master's Total

CIP: 130603

Female

Female Total

1

1

2

2

0

6

3

6

7

3

10

9

7

21

6

0 5 10 15 20 25

2012‐13

2013‐14

2014‐15

2015‐16

2016‐17

Graduate Admissions ‐ Masters Degree

Applied Accepted Enrolled



Table 8: Graduate Application / Enrollment Funnel ‐ Doctoral Degree


d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d


Hispanic/Latino 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


Asian 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Black 2 0 0 0 0 0 2 2 0 0 0 0 0 0 0


White 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


12 5 1 12 9 0 6 3 1 10 5 3 2 1 0


Hispanic/Latino 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0


Asian 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Black 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0


White 2 0 0 1 0 0 0 0 0 1 1 0 0 0 0

Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


11 5 1 12 6 3 4 3 1 6 3 0 8 4 123 10 2 24 15 3 10 6 2 16 8 3 10 5 1


2016‐172012‐13 2013‐14 2014‐15 2015‐16

Doctorate Total

CIP: 130603

Female

Female Total

Male

Male Total

2

3

2

3

1

10

15

6

8

5

23

24

10

16

10

0 5 10 15 20 25

2012‐13

2013‐14

2014‐15

2015‐16

2016‐17

Graduate Admissions ‐ Doctoral Degree




Table 8: Graduate Application / Enrollment Funnel ‐ Masters and Doctoral Degrees


d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d

Applie

d

Accep

ted

Enrolle

d


Hispanic/Latino 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


Asian 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Black 2 0 0 0 0 0 2 2 0 1 0 0 0 0 0


White 1 0 0 0 0 0 1 1 0 0 0 0 1 1 0

Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


19 10 2 18 12 1 10 6 2 19 7 4 6 3 0


Hispanic/Latino 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0


Asian 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0

Black 1 0 0 0 0 0 1 1 0 1 1 1 1 0 0


White 3 1 0 1 0 0 1 1 0 3 2 0 1 1 0

Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


14 6 1 15 6 3 7 6 2 18 8 1 10 5 133 16 3 33 18 4 17 12 4 37 15 5 16 8 1


2012‐13 2013‐14 2014‐15 2015‐16 2016‐17

Male Total

CIP: 130603

Grand Total

Female

Female Total

Male

3

4

4

5

1

16

18

12

15

8

33

33

17

37

16

0 5 10 15 20 25 30 35 40

2012‐13

2013‐14

2014‐15

2015‐16

2016‐17

Graduate Admissions ‐ Masters and Doctoral Degree



Educational Psychology & Learning Systems | Measurement and Statistics Program Table 9: Graduate Program Enrollment Trends

Graduate Headcount Enrollment by Degree Level Sought

Gender Race/Ethnicity M D M D M D M D M D M D M D


Asian 1 0 0 1 0 1 0 1 0 1 0 1 0 1

Black 0 1 0 1 0 1 1 1 1 1 1 1 0 1

Hispanic 0 0 0 0 0 0 0 0 0 0 0 0 0 0


White 0 1 0 1 1 1 1 1 0 0 0 0 0 0


Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Not Reported 0 0 0 0 0 0 0 0 0 0 0 0 0 0

3 16 4 13 6 13 7 14 6 13 5 12 2 10


Asian 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Black 0 1 0 1 0 1 0 1 0 1 1 0 1 0

Hispanic 1 0 1 0 0 0 0 0 0 0 0 0 0 0


White 2 4 2 5 2 4 1 4 1 3 0 1 0 0


Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Not Reported 0 0 0 0 0 0 0 0 0 0 0 0 0 0

3 10 4 13 4 13 1 17 2 16 3 13 2 10

6 26 8 26 10 26 8 31 8 29 8 25 4 20


Source: Fall Preliminary Student Instruction Files (SIFP)

Fall Fall

2010 2011 2015 2016

Fall Fall Fall Fall Fall

Female

Male

2012 2013 2014

33 24

Degree Level Total

Grand Total

Female Total

Male Total

32 34 36 39 37

68

108 8 8

4

26

26

26 3129

25

20

0

5

10

15

20

25

30

35

40

45

2010 2011 2012 2013 2014 2015 2016

Master's Doctorate



Table 9: Graduate Program Degrees Awarded Trends

Graduate Degrees Awarded by Level

Gender Race/Ethnicity M D M D M D M D M D M* D


Asian 1 0 1 1 0 0 0 0 0 0 0 0

Black 0 0 0 0 0 0 0 0 0 0 0 1

Hispanic 0 0 0 0 0 0 0 0 0 0 0 0


White 0 0 0 0 0 0 1 0 0 1 0 0


Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0

Not Reported 0 0 0 0 0 0 0 0 0 0 0 0

4 2 2 2 2 0 2 1 4 4 3 4


Asian/Pacific Islander 0 0 0 0 0 0 0 0 0 0 0 0

Black 0 0 0 0 0 0 0 0 0 0 0 0

Hispanic 0 0 1 0 0 0 0 0 0 0 0 0


White 0 0 1 0 1 0 0 1 0 0 0 2


Two or More Races 0 0 0 0 0 0 0 0 0 0 0 0

Not Reported 0 0 0 0 0 0 0 0 0 0 1 0

3 0 3 1 2 0 0 2 0 0 3 4

7 2 5 3 4 0 2 3 4 4 6 8


* Master's includes one conferred Specialist’s degree

Source: Student Instruction Files ‐ Degrees (SIFD)

2012‐13 2013‐14 2014‐15 2015‐16

Grand Total

2010‐11

9 8 14

Male Total

Degree Level Total

Female

Female Total

Male

8 4 5

2011‐12

7

54

2

4

6

2

3

03

4

8

0

2

4

6

8

10

12

14

16

2010‐11 2011‐12 2012‐13 2013‐14 2014‐15 2015‐16

Master's Doctorate

QER Manual 2017

Florida State University

Doctoral Completion Tables – 2017-18

Time to Degree – Doctoral

Graduating

Year

3

years

or less

4

years

5

years

6

years

7

years

8

years

9

years

10

years

Number

still

enrolled

after 10

years

2016-17 1 1

2015-16 3 3 1

2014-15 2 1 1

2013-14 1 1 1 2

2012-13

2011-12 2

2010-11 1 2

Funding Sources for Graduate Student Support – Fall Term

Doctoral

2017* 2016 2015 2014 2013

Scholarships

University Fellowship 2 2 1 1

External Fellowship 0 0 0 0

Scholarship 0 0 0 0

Percent on Scholarship 10% 6% 3% 3%

Assistantship

E&G Funded 8 9 5 8

Foundation Funded 0 0 0 0

Distance Learning Funded 0 0 0 0

C&G/External 3 5 11 9

Percent on Assistantships 52% 44% 52% 55%

Self-Funded

Self-Funded 8 16 14 13

Percent Self Funded 38% 50% 45% 42%

Total Number of Students 21 32 31 31

Masters

2017* 2016 2015 2014 2013

Scholarships

University Fellowship 0 0 0 0

External Fellowship 0 0 0 0

Scholarship 0 0 0 1

Percent on Scholarship 0 0 0 11%

Assistantship

E&G Funded 0 1 2 2

Foundation Funded 0 0 0 0

Distance Learning Funded 0 0 0 0

C&G/External 1 1 1 0

Percent on Assistantships 25% 20% 27% 22%

Self-Funded

Self-Funded 3 8 8 6

Percent Self Funded 75% 80% 73% 67%

Total Number of Students 4 10 11 9

*Data for Fall 2017 are currently unavailable.

quality enhancement review august 2017 … · in fall 2015, zhang joined the program at the rank of...

Documents