1

2

Evaluation Report

A preliminary report to the faculty and administrators

of the online distance learning programin the Department of Educational Research

in the College of Education at Florida State University.April, 2002

“There is no royal road to learning”Euclid, 300 B.C.

3

Agenda

I. Evaluation Team

II. Description of the Students

III. Procedures Used

IV. Results

V. Conclusions

VI. Preliminary Recommendations

VII. Reactions – Discussion and Written

4

Key Evaluation Results

1. Overall, How Would You Rate the Program?

2. Overall, How Would You Rate the Curriculum?

3. Overall, How Would You Rate the Faculty?

M = 3.29

SD = .68

M = 3.29SD = .72

M = 3.49SD = .68

(4 = Excellent)

5

I. The Evaluation Team

6

Michael Simonson, Ph.D.

Evaluation Coordinator

7

Judy Converso, Ph.D.

Comparisons WithSimilar University Programs

8

Maggie CrawfordInformation Specialist

Telephone Interviews

9

II. The Students

10

Descriptive Statistics

41 41.24 9.40

41 14.35 10.25

39 5.77 4.16

38 9.97 6.18

36

Age

Years Since BA/BS

Number of OnlineCourses Taken andTaking

Hours/Week Studying fora 3-Credit Online Course

Valid N (listwise)

N Mean Std. Deviation

11

56

53

51

50

49

48

47

45

44

43 41

40

39

38

37

36

35

32

31

29

28

25

24

Ages of Students Responding

Median = 41

12

15

13

12

11

10

9

7

6

5

4

3

2

1

0

Missing

Median = 6

Number of Online Courses Taken + Taking

13

25

20

15

12

10 8

6

5

0

0

0

0

Missing

Hours/Week Studying – One Course

Median = 10 hours/week

14

III. Procedures

1. Student Survey (online)

2. Telephone Interviews

3. Faculty Interviews and Meetings

4. Literature Searches and Program Comparisons

5. “Lurking”

15

IV. ProceduresV. Conclusions

16

1. Student Survey

Response = ~70%

17

1. A. Are You Satisfied with Your Courses and Curriculum?

1. Grading Requirements? 1 3.3

2. Required Course Content? 1 3.2

3. Required Course Selection? 1 3.2

4. Course Difficulty? 1 3.2

5. Elective Course Content? 10 3.1

6. Relevance of Courses ? 2 3.1

7. Online Teaching? 2 3.0

8. Elective Course Selection? 10 2.9

Not Satisfied Mean

N = 414 = Excellent

18

1. B. Are You Satisfied with Your Instructors?

1. Content Knowledge? 1 3.6

2. Feedback? 3 3.2

3. Skills at Online Teaching? 1 3.1

4. Interaction? 3 2.9

Not Satisfied Mean

N = 414 = Excellent

19

1. C. Are You Satisfied With?

1. Number of Required Courses? 2 3.4

2. FSU Technical Support 43.2

3. Prep. For Profession? 6 3.1

4. Portfolio Experience? 10 3.1

5. Number of Electives? 5 3.1

6. FSU Library Support? 9 2.9

Not Satisfied Mean

N = 414 = Excellent

20

1. D. General Questions: How Would You Rate:

1. Faculty? 3.5

2. Program – Overall? 3.3

3. Curriculum? 3.3

4. FSU’s Support? 3.3

5. Texts and Printed Materials? 3.1

6. Program Support Services 3.0

Mean

N = 414 = Excellent

21

 Open-Ended Questions-Student Survey What Improvements Would You

Recommend?

• More electives

• More teaching and less discussion

• More courses each semester

• Improve Blackboard

• Eliminate discrepancies in course policies

22

Open-Ended Questions-Student Survey What Do You Like Best?

• Content

• Faculty

• Reputation of the program

• Freedom

• Flexibility

• Relevance

23

Open-Ended Questions-Student Survey Why Is Your Favorite Course Your

Favorite? • Quality interaction with the instructor

• Excellent professor

• Interesting Content

• Various instructional Strategies used

• Relevant content

24

Open-Ended Questions-Student Survey How Likely Would You Be To Choose An

Online Program, Given Free Choice?

Almost evenly split between those who prefer face-to-face and online learning

25

Open-Ended Questions-Student Survey Other Comments?

• Do something about Blackboard

• Offer courses in the summer

• Provide more interaction with faculty

26

2. Telephone Interviews

• 10 Students Called ~ Randomly• 5 Females and 5 males (30 Females/18 Males)

• Age = ~45

• 15 – 45 Minutes Each

• 3 Questions

27

Summary Statements – Telephone Interviews

Why did you enroll?

• Reputation of the program

• Convenience

• Content

• Skill Building

28

Summary Statements – Telephone Interviews

Major Program Strength?

• Faculty

• Organization

• Flexibility

• Students

29

Summary Statements – Telephone Interviews

What Changes Would You Recommend?• Get a better system than Blackboard• Improve consistency of policies between courses• More immediate feedback• Help us connect with other students• Be realistic about time commitments• Consider synchronous instruction

30

3. Faculty Interviews

• 7 Interviews

• Face-to-face and Telephone

• 30 – 60 Minutes

31

Summary Statements

Faculty Interviews • Instructional Systems Design is the key to the program• Rigor and quality are critical• The program is very labor intensive• Synchronous instruction should be considered• Various instructional platforms should be available• Identify program benchmarks• Provide training in distance education for faculty• Support for faculty is needed – graduate assistants and local

production equipment• Camaraderie among students is important – add synch.• 3 faculty promised, only one hired

32

4. Literature & Program Review

• 11 programs reviewed

• Titles of courses compared

• Similarities and differences noted

33

VI. Preliminary Recommendations

Policy & Management

Faculty & Curriculum

Instruction & Delivery

34

RecommendationsPolicy & Management

• Tuition and Fees

• Program Policy Statements

• Incentives and Involvement of Faculty

• Entrance Requirements – Skills AND Resources

• 5-Year Plan

35

RecommendationsFaculty & Curriculum

• Move from text-based instruction to media

• Explore various management platforms in addition to Blackboard

• Review the capstone experience

• Consider specializations w/in the degree

• Involve full-time faculty

36

RecommendationsInstruction & Delivery

• Offer Summer courses• Increase selection of online materials• Consider use of adjuncts, supervised by

full-time faculty• Secure local production resources for

faculty• Offer synchronous instruction, and consider

face-to-face activities

37

VII. Reactions

38