edf 711: survey research...seminar evaluations have been lacking in providing specific feedback to...

1

Marshall University Doctoral Seminar Evaluation

Project Completed as Course Requirement in

EDF 711: Survey Research

Summer 2015

By: Laura McCullough, Pamela Meadows, and

Rebecca Metzger

Marshall University

College of Education and Professional Development

November 22, 2015

2

Abstract

The purpose of this study was to determine the level of satisfaction with the Marshall

University College of Education and Professional Development Doctoral Seminar. There is

currently no body of research regarding the development of the doctoral seminar and the

doctoral seminar planning committee identified a need for a formal evaluation of the seminar.

The study consisted of an individual session survey and overall seminar survey containing

quantitative and qualitative questions to measure student and faculty satisfaction. The results

from this study will be utilized by the doctoral seminar planning committee to identify areas for

improvement and aid in the scheduling and recruitment of speakers for future seminars.

3

Table of Contents

Introduction ..................................................................................................................................................... 4

Statement of the Problem .............................................................................................................................. 4

Purpose of Study ........................................................................................................................................... 5

Research Questions ....................................................................................................................................... 5

Survey Design/Methods .................................................................................................................................. 6

Findings ........................................................................................................................................................... 7

Participant Characteristics ............................................................................. Error! Bookmark not defined.

Seminar Logistics…………………………………………………………………………………….……9

Presentation Quality………………………………………………...…………………..…………..……10

Keynote Speaker……………………………………………………………………………………………….………………………………11

Conclusions…………………………………………………………………………………………………11

Discussion/Implications/Recommendations ............................................................................................... 12

Appendix A…………………………………………………………………………………………………16

Consent……………………………………………………………………………………………….….16

Appendix B…………………………………………………………………………………………………17

Individual Session Evaluation Survey Instrument……………………………………………………....17

Appendix C………………………………………………………………………………………………...18

Overall Doctoral Seminar Evaluation Survey Instrument………………………………………………18

Appendix D………………………………………………………………………………………………...19

IRB Approval Letter………………………………………………….………………………………...19

Appendix E………………………………………………………………………………..……………….20

Data: Participant Characteristics……………………………………………………………………….20

Appendix F………………………………………………………………………………………………...22

Data: Overall Doctoral Seminar Evaluations…………………………………………………………..22

Appendix G………………………………………………………………………………………………..28

Data: Individual Session Evaluations………………………………………………………………….28

4

Introduction

Doctoral programs have long histories of requiring students to complete residencies as a

component of their studies. Residencies provide students with the opportunity to build

meaningful relationships with their peers and staff to further develop research and professional

skills. When Marshall University was granted permission to offer a doctorate in education

degree, the program sought students who were full-time professionals who desired to earn an

advanced degree as a part-time student. Residency requirements changed to meet the needs of

the student population.

Doctoral students are expected to participate in an annual seminar, scheduled on a

Saturday each fall. Seminars are coordinated by a group of doctoral students and a faculty

member, and students who assist in coordinating the seminar or presenting at a session use their

participation as an artifact in their doctoral portfolio. Students who participate in a more inactive

role use the seminar as a way to increase their knowledge about the program and develop

relationships with faculty that may lead to the identification of a chairperson and committee.

For over ten years, the doctoral seminar at the Marshall University College of Education

and Professional Development has provided students with networking opportunities, while

guiding them through the various stages of the doctoral program. The doctoral seminar planning

committee has welcomed assistance with the development of an evaluation system for the

seminar. The purpose of this study was to evaluate student satisfaction with regard to the annual

doctoral seminar.

Statement of the Problem

Participation in the annual seminar is a requirement for doctoral students. The seminar

planning committee needs to evaluate student and faculty satisfaction with individual sessions

5

and the overall seminar to determine if the training is meeting residency goals. Previous

seminar evaluations have been lacking in providing specific feedback to the planning

committee, and inconsistencies with surveys were also present. Research is needed to

determine the satisfaction level of doctoral students and faculty with the seminar and to

provide information to the planning committee to improve future seminars. The development

of session and overall seminar surveys can also provide baseline data for future training.

Purpose of Study

The purpose of this study was to determine the satisfaction level of doctoral students and

faculty with their experiences in attending the Marshall University College of Education and

Professional Development 2015 Doctoral Seminar. The results of this study will identify areas

for improvement and will be utilized by the doctoral seminar planning committee for the

development of future doctoral seminars at Marshall University Graduate College. Data

collected regarding individual session evaluations will aid speakers in improvement of

presentation skills, development of their curriculum vitae or resume, and will serve as

documentation required for doctoral portfolio, promotion, or tenure.

Research Questions

The following specific questions were used to guide the study:

1. What are the demographics of seminar attendees?

2. How do seminar attendees rate the effectiveness of the registration process?

3. How do attendees rate the seminar sessions in meeting individual and professional

needs?

4. How do attendees rate the quality of seminar training sessions?

5. How do seminar attendees rate networking opportunities?

6

6. How do attendees rate their satisfaction with facilities and refreshments?

7. What do attendees identify as the major benefits of the seminar?

8. What recommendations do attendees have for improving speaker performance

and the overall seminar?

Survey Design/Methods

A total of sixteen sessions were offered during the Marshall University College of

Education and Professional Development 2015 Doctoral Seminar. Guidance from the seminar

planning committee regarding evaluation criteria and program needs was utilized to develop two

survey instruments. The survey instruments addressed attendee level of satisfaction with various

seminar logistics, topics, and speaker effectiveness.

Current doctoral students in a survey design course and the doctoral seminar planning

committee assisted in validating the surveys. Session evaluation surveys were color-coded for

organization following the seminar, and collection boxes were placed in session rooms to

receive anonymous surveys at the end of the sessions. Four moderators were responsible for

administering an anonymous paper survey to all session attendees within the last five minutes

of the individual sessions, and seminar overall evaluation surveys were provided five minutes

prior to the last session ending. No identifying information was obtained from participants.

The seminar session evaluation surveys asked respondents to identify the session

attended and the level of expertise in the subject area prior to attending the session using a

five-point Likert scale. Participants were asked seven quantitative questions to rate the

quality of the training session using a seven-point Likert scale. The questions focused on the

organization of the presentation, effective use of time, visual aids, and responsiveness of the

presenter. Participants were also asked to rate their agreement in recommending the presenter

7

for future seminars and in session content contributing to their overall knowledge. Two

qualitative questions asked respondents to identify areas of growth as a result of the session

and recommendations to aid the presenter(s) for future sessions.

The demographic section of the seminar overall evaluation consisted of one question

to identify the current status of the respondent in the doctoral program as a program graduate,

faculty, guest, or current student. Current students were asked to identify their program status

as currently completing regular coursework, admitted to candidacy/prospectus not approved,

and admitted to candidacy/prospective approved. Students who identified themselves as

currently completing regular coursework were asked to approximate the number of hours they

had completed in the doctoral program.

A total of 10 quantitative questions on the overall evaluation addressed satisfaction

with the registration process, session topics, facilities, and refreshment using a seven-point

Likert scale. Respondents were also asked to gage if their understanding of the doctoral

program and procedures significantly increased as a result of the attending the seminar. The

qualitative questions asked participants to identify the major benefits of attending the seminar

and suggestions to improve the seminar.

Findings

The primary purpose of this survey was to evaluate the perceptions of the doctoral

students, faculty, and guests who participated in the 2015 Marshall University Doctoral seminar.

Specifically, the questions asked were to measure the demographic characteristics of the

participants, logistics of the seminar, and quality of the presentations. There was also one item

polling if the participants believed having a keynote speaker for next year’s seminar would

improve their experience. Examples of the two instruments used are provided in Appendix B and

Appendix C. Detailed survey responses were organized by question and are provided in

8

Appendices E, F and G.

Participant Characteristics

Thirty people who attended the Marshall University Doctoral Seminar completed an

overall conference evaluation survey (n=30). Twenty-eight (93.3%) of evaluation completers

were current students, one was a guest, and one was a program graduate. The majority of the

current students (67.9%) who completed an overall seminar evaluation were completing course

work. Seven respondents (25%) reported they were either admitted to candidacy or that their

prospectus was not yet approved. Two respondents (7.1%) reported that they were admitted to

candidacy or that their prospectus was approved.

The frequency of the credit hours completed by the students completing course work was

not normally distributed. Eight current students, representing 47.2% of the respondents, had

completed fewer than 10 credit hours of course work. One current student, representing 5.9% of

the total respondents, reported having completed 18 credit hours. Two current students,

representing 11.8% of the respondents, had completed more than 20, but less than 29 credit

hours. Four current students, representing 23.6% of the respondents, had completed a range from

30 to 39 credit hours. One student, representing 5.9% of the respondents, had completed 43

credit hours. One student, representing 5.9% of the respondents, had completed 56 credit hours.

The percentages reflected above are “valid percentages,” because 13 current students neglected

to report how many credit hours they had already completed. A summary of participant

characteristics is located in Appendix. E.

One hundred, eighteen duplicated seminar session attendees self-reported their expertise

before attending each of the seminar sessions they chose to attend. The most frequent response

9

(33%) from attendees indicated an intermediate level of prior expertise. Twenty-five percent

perceived their prior expertise as basic. Sixteen percent judged their prior expertise to be

between intermediate and advanced. Fourteen percent believed their prior expertise was between

basic and intermediate. The percentage of students who had advanced levels of expertise prior to

attending one of the seminar’s sessions was the lowest at 11%.

Seminar Logistics

All of the attendees who completed the doctoral seminar evaluation either somewhat

agreed, agreed, or strongly agreed the registration process was organized and 93% agreed the

registration process was efficient (93%). Seminar participants either somewhat agreed, agreed, or

strongly agreed the seminar was relevant to the attendee’s programmatic needs (97%) and was

relevant to their individual needs (97%). At 97%, participants believed the sessions were of the

appropriate length of time, and that networking opportunities were available. One hundred

percent of the people who completed the doctoral seminar evaluation believed the facilities met

their needs. Eighty-seven percent of the people who completed the doctoral seminar evaluation

thought that the food and beverages were satisfactory and that their understanding increased as a

result of attending the seminar.

Eighteen people who completed the doctoral seminar evaluation made positive statements

for major benefits of attending the doctoral seminar. There were three themes: (1) useful for

gathering information about the future steps in their program; (2) enjoyed networking with

others; and, (3) general compliments on the seminar. There were only three suggestions for

improvement: (1) information from other sessions would be helpful; (2) please make sure

everyone is informed about the date; and, (3) shorten.

10

The findings from the overall doctoral seminar evaluation survey were overwhelmingly

positive (Appendix F). The only limitation is that the number of people completing this

instrument was small. According to registration records, a total of 58 people attended the

doctoral seminar.

Presentation Quality

Participants completed evaluations regarding each of the seminar’s presentations they

attended. There was a range of 130 to 131 responses to the items being measured. The item with

the highest mean score (6.84 with a standard deviation of .408) measured the presenter’s

responsiveness to the audience’s questions. Presenters ranked highly in items related to their

knowledge (6.78 mean with a standard deviation of .469), efficient use of time (6.70 mean with a

standard deviation of .591), organized presentation (6.60 mean with standard deviation of .731),

and style of presenting and visual aids (6.58 mean with a standard deviation of .794). The

participants were likely to recommend the speakers for future sessions (6.75 mean with a

standard deviation of .62). They also believed that the sessions contributed to their overall

knowledge of the subject (6.66 mean with a standard deviation of .73). The mean scores varied

slightly, ranging from 6.58 to 6.84. The two items with the most variance measured presentation

style and visual aids and contribution to overall knowledge.

Reflection on the Portfolio Process was the most popular session with 15 attendees

(11.5% of the population). Current Issues C&I with Dr. Edna Meisel was the least-attended

session with 4 (3.1%) participants. This session, however, along with IRB, Makey-Makey, and

Library Skills received a perfect rating of seven. The lowest rated seminar session was Current

11

Issues: Higher Education and K-12 (mean 6.0 with a standard deviation of 1.291). To view a

distribution table illustrating the attendance and a table showing the mean score for each seminar

session held, see Appendix G.

There were several reoccurring patterns in the responses to the open-ended items on the

seminar session evaluation, most of them were positive. In the areas of growth item, 25

participants noted that their knowledge increased as a result of the session. Nine people said the

sessions gave them some sort of guidance. Six people offered praise to the presenter. Three

people felt encouraged after attending a session. There were also miscellaneous comments in this

field. In the recommendations to the speaker item, 16 offered praise to the presenter. Four

participants wanted the session to be longer for either more content (2) or for more time for

discussion (2). Two people offered the presenter suggestions for improvement. For all

comments, please see Appendix G.

Keynote Speaker

Fifty-nine valid percent (one response was missing) of the people who completed the

doctoral seminar evaluation believed that having a keynote speaker would improve the

experience. Fourteen valid percent of respondents either disagreed or, strongly disagreed. Thirty-

seven valid percent of respondents neither agreed nor disagreed with the statement regarding a

keynote speaker.

Conclusions

Overall, the majority of attendees rated the doctoral seminar favorably. The registration

process was well-received by attendees. Those who attended the seminar expressed that both

12

their individual and programmatic needs were met. The information gathered from the surveys

also indicated that networking and the opportunity to learn more about their program of study

were beneficial. Attendee evaluation of presenters was also positive and the majority of

attendees are in favor of the addition of a keynote speaker in the future.

While the general viewpoint of attendees demonstrated satisfaction with the seminar,

there were a few areas where the need for improvement was indicated. Attendees noted slight

dissatisfaction with regard to food and beverage selection, however, the overall viewpoint was

positive for this element of the seminar. The open-ended questions revealed additional areas of

suggestion for improvement, including the sharing of content for all sessions, timely and

accurate dissemination of seminar date, and decreasing the length of the seminar. Examination of

individual session evaluations was also positive; however, answers demonstrated greater

variance for questions regarding presentation style and visual aids, as well as the contribution of

session information to the overall knowledge of attendees.

Discussion/Implications/Recommendations

Overall and individual session evaluations indicated that attendees were highly satisfied

with the doctoral seminar. There appeared to be a fair distribution of attendees, ranging from

recently admitted students to graduates. Evaluation data revealed that attendees found various

aspects of the seminar to be helpful and enlightening.

According to the administrative assistant for Leadership Studies, one-hundred thirty-four

invitations for the doctoral seminar were sent to current students, as well as recent graduates.

Registration records, however, indicated that only fifty-eight people attended the doctoral

seminar. Using the information given, it can be deducted that less than half of those individuals

13

invited actually attended the seminar. While current students are required to attend at least three

doctoral seminars, graduates are invited to receive awards in recognition of their academic

achievements.

Poor attendance among current students is a primary concern with regard to the planning

of future doctoral seminars. The number of attendees per session (Appendix G) indicates that

some sessions were poorly attended. It should also be noted that although fifty-eight people

attended the seminar, only thirty completed the overall seminar evaluation, which was

disseminated during the final session of the day. This finding suggests that although individuals

are coming to the seminar, many are not attending through the end of the day. It is possible that

some attendees are leaving after the lunch break, during which time they are required to meet

with their committee chairs. To ensure that attendees stay for the duration of the seminar, it is

recommended that the seminar planning committee develop a plan to ensure accountability of

students with regard to attendance. Further examination of individual session attendance would

reveal the approximate number of people who leave before the conclusion of the seminar.

Based on information from the evaluations, it is evident that attendees most appreciate

the opportunities that the doctoral seminar allows for networking with fellow students and

faculty. By providing students with an avenue for interacting with other students at various

stages within the program, students become more comfortable with the expectations of the

program. Networking also allows for the development of friendships and peer-relationships and

can be a key factor with regard to emotional support, which plays a pivotal role in the success of

graduate students. Sessions based on the critical aspects of the doctoral program, including

introduction to doctoral studies and mock portfolio and dissertation defenses, offer students

guidance with regard to critical information needed to ensure timely completion of the doctoral

14

program.

The inclusion of more sessions that allow for discussion among students, and between

students and faculty, is recommended for future seminars. The doctoral seminar is often the only

chance that students have to meet faculty members. By creating more opportunity for

networking, students will not only gain a better understanding of program requirements, but they

will also be equipped with the information needed to aid them in making a more informed

decision in regard to the selection of a committee chair.

While eighty-seven percent of respondents indicated that they were satisfied with the

food and beverage selection, slight room for improvement may be indicated. It may be

advantageous for the planning committee to survey attendees prior to future seminars to

determine meal preferences. While it is impossible to satisfy all attendees, a short meal

preference survey would help to identify the types and quantities of food needed, while

eliminating waste.

Other suggestions for improvement include the inclusion of a keynote speaker,

availability of session presentations, and re-examination of seminar length. A majority of

attendees were in favor of the addition of a keynote speaker. It is recommended that the

planning committee consider recruiting an enthusiastic former graduate or general motivational

speaker to offer a keynote address at future seminars. The inclusion of a general session keynote

would also eliminate the need for recruitment of speakers for one time block. As suggested by

attendee comments, it would be helpful for presenters to post their presentations in an online

repository where students could review or retrieve the learning materials from sessions that they

were unable to attend. For future doctoral seminar evaluations, it may also be beneficial to

15

include a question regarding the efficacy of the award ceremony. Although data was not

collected regarding the awards ceremony at this seminar, general attendance appeared low

among both current students and graduates. The shortening or elimination of the award

ceremony would decrease the total number of hours required for attendance. Another option

would be to combine a general keynote with the award session, while offering three additional

time blocks of sessions. This option would fulfill attendees’ desires for a keynote speaker, while

still including the award ceremony and decreasing the overall seminar time by one hour.

16

Appendix A: Consent Form

Doctoral Seminar Evaluation Anonymous Survey Consent

You are invited to participate in a research project entitled Marshall University Doctoral Seminar

Evaluation designed to analyze the experiences of students, faculty, and guests attending the doctoral

seminar at Marshall University Graduate College. The study will evaluate the quality of individual

sessions, as well as the overall seminar. Dr. Ronald Childress is the principal investigator, and this

research is being conducted as part of the Survey Research in Education Course requirements for Pamela

Meadows (co- principal investigator), Laura McCullough (co-principal investigator), and Rebecca

Metzger (co-principal investigator).

Each survey is comprised of twelve brief questions that may take between 1 to 3 minutes to complete.

Your replies will be anonymous, so do not put your name anywhere on the survey. There are no known

risks involved with this study. Participation is completely voluntary, and there will be no penalty or loss

of benefits if you choose to not participate in this research study or to withdraw. You may choose to not

answer any question by simply leaving it blank. Submitting the paper survey indicates your consent for

use of the answers you supply. If you have any questions about the study you may contact Dr. Ronald

Childress at (304) 746-1904, Pamela Meadows at (304) 550-4439, Laura McCullough at (304) 546-5372,

or Rebecca Metzger at (304) 419-4453.

If you have any questions concerning your rights as a research participant you may contact the Marshall

University Office of Research Integrity at (304) 696-4303.

By completing this survey and submitting it, you are also confirming that you are 18 years of age or older.

Please keep this page for your records.

Thank you.

Pamela Meadows

Laura McCullough

Rebecca Metzger

17

Appendix B: Individual Session Evaluation Survey Instrument

Marshall University Doctoral Seminar Session Evaluation-Fall 2015

Please circle the name of the session you attended.

session title 1 session title 2 session title 3 session title 4

On the scale below, indicate your level of expertise in this subject area

prior to this session:

(Basic) (Intermediate) (Advanced)

1 2 3 4 5

Using the scale of 1-7 below, please rate the quality of the training session by circling the number that

corresponds to your opinion.

Strongly Disagree

Disagree Somewhat Disagree

Neither Agree or Disagree

Somewhat Agree

Agree Strongly

Agree

The presentation was well organized. 1 2 3 4 5 6 7

The speaker effectively utilized the allotted time.

1 2 3 4 5 6 7

The speaker presentation style & visual aids were effective.

1 2 3 4 5 6 7

The speaker(s) was knowledgeable about the topic.

1 2 3 4 5 6 7

The speaker(s) was responsive to audience questions.

1 2 3 4 5 6 7

I would recommend this speaker for future seminars.

1 2 3 4 5 6 7

The session contributed to my overall knowledge of the topic.

1 2 3 4 5 6 7

COMMENTS:

My areas of growth resulting from the session are:

Recommendations to aid the speaker(s) for future sessions:

18

Appendix C: Overall Doctoral Seminar Evaluation Survey Instrument

Marshall University Doctoral Seminar Overall Evaluation - Fall 2015

Please indicate your current status in the doctoral program:

_____ Program Graduate

_____ Faculty

_____ Guest

_____ Current Student (please identify your program progress below):

_____ Currently completing regular coursework

Approximate Number of Hours Completed _____

_____ Admitted to Candidacy/Prospectus Not Approved

_____ Admitted to Candidacy/Prospectus Approved

Using the scale provided, please evaluate the following elements of the doctoral seminar.

Strongly Disagree

Disagree Somewhat Disagree

Neither Agree or Disagree

Somewhat Agree

Agree Strongly Agree

The seminar registration process was organized. 1 2 3 4 5 6 7

The seminar registration process was efficient. 1 2 3 4 5 6 7

Session topics were relevant to my programmatic needs.

1 2 3 4 5 6 7

Session topics were relevant to my individual needs.

1 2 3 4 5 6 7

Session lengths of 50 minutes were appropriate. 1 2 3 4 5 6 7

Opportunities to network were provided. 1 2 3 4 5 6 7

The facilities met my needs. (room size, seat availability, heating/cooling, etc.)

1 2 3 4 5 6 7

The food and beverage selection was satisfactory. 1 2 3 4 5 6 7

My understanding of the doctoral program & procedures significantly increased as a result of attending the seminar.

1 2 3 4 5 6 7

A keynote speaker would improve the seminar experience.

1 2 3 4 5 6 7

COMMENTS:

The major benefits of attending the seminar were:

What suggestions do you have for improving the doctoral seminar?

19

Appendix D: IRB Approval Letter

20

Appendix E: Participant Characteristics

Statistics

Attendee Status Student Status

Credit Hours

Completed

N Valid 30 28 17

Missing 0 2 13

Mean 3.87 1.39 20.59

Std. Deviation .571 .629 16.440

Variance .326 .396 270.257

Skewness -4.782 1.398 .622

Std. Error of Skewness .427 .441 .550

Sum 116 39 350

Percentiles 10 4.00 1.00 3.20

Attendee Status

Frequency Percent Valid Percent

Cumulative

Percent

Valid Program Graduate 1 3.3 3.3 3.3

Guest 1 3.3 3.3 6.7

Student 28 93.3 93.3 100.0

Total 30 100.0 100.0

21

Appendix E: Summary of Participant Characteristics (continued)

Student Status


Cumulative

Percent

Valid Currently Completing Course

Work

19 63.3 67.9 67.9

Admitted to Candidacy or

Prospectus Not Approved

7 23.3 25.0 92.9

Admitted to Candidacy or

Prospectus Approved

2 6.7 7.1 100.0

Total 28 93.3 100.0

Missing System 2 6.7

Total 30 100.0

Credit Hours Completed


Cumulative

Percent

Valid 0 1 3.3 5.9 5.9

4 1 3.3 5.9 11.8

6 2 6.7 11.8 23.5

7 2 6.7 11.8 35.3

9 2 6.7 11.8 47.1

18 1 3.3 5.9 52.9

21 1 3.3 5.9 58.8

28 1 3.3 5.9 64.7

30 1 3.3 5.9 70.6

34 1 3.3 5.9 76.5

36 2 6.7 11.8 88.2

43 1 3.3 5.9 94.1

56 1 3.3 5.9 100.0

Total 17 56.7 100.0


Total 30 100.0

22

Appendix F: Overall Doctoral Seminar Evaluations

Registration Was Organized


Cumulative

Percent

Valid Neither Agree or Disagree 1 3.3 3.3 3.3

Somewhat Agree 1 3.3 3.3 6.7

Agree 10 33.3 33.3 40.0

Strongly Agree 18 60.0 60.0 100.0

Total 30 100.0 100.0

Registration Was Efficient


Cumulative

Percent



Agree 9 30.0 30.0 40.0

Strongly Agree 18 60.0 60.0 100.0

Total 30 100.0 100.0

Relevant to Programmatic Needs


Cumulative

Percent



Agree 10 33.3 33.3 53.3

Strongly Agree 14 46.7 46.7 100.0

Total 30 100.0 100.0

23

Appendix F: Overall Doctoral Seminar Evaluations (continued)

Relevant to Individual Needs


Cumulative

Percent



Agree 8 26.7 26.7 46.7

Strongly Agree 16 53.3 53.3 100.0

Total 30 100.0 100.0

Sessions were for the Appropriate Length of time


Cumulative

Percent

Valid Disagree 1 3.3 3.3 3.3

Neither Agree or Disagree 1 3.3 3.3 6.7


Agree 7 23.3 23.3 36.7

Strongly Agree 19 63.3 63.3 100.0

Total 30 100.0 100.0

Networking Opportunities were Available


Cumulative

Percent

Valid Somewhat Disagree 1 3.3 3.3 3.3


Agree 7 23.3 23.3 40.0

Strongly Agree 18 60.0 60.0 100.0

Total 30 100.0 100.0

24


Facilities met my needs


Cumulative

Percent

Valid Agree 10 33.3 33.3 33.3

Strongly Agree 20 66.7 66.7 100.0

Total 30 100.0 100.0

Food and Beverages were satisfactory


Cumulative

Percent



Agree 7 23.3 23.3 36.7

Strongly Agree 19 63.3 63.3 100.0

Total 30 100.0 100.0

Understanding Increased


Cumulative

Percent



Agree 10 33.3 33.3 63.3

Strongly Agree 11 36.7 36.7 100.0

Total 30 100.0 100.0

25


Keynote Speaker would improve the experience


Cumulative

Percent

Valid Strongly disagree 1 3.3 3.4 3.4

Disagree 3 10.0 10.3 13.8



Agree 5 16.7 17.2 69.0

Strongly Agree 9 30.0 31.0 100.0

Total 29 96.7 100.0


Total 30 100.0

26


Major Benefits of Attending


Cumulative

Percent

Valid 18 60.0 60.0 60.0

Better than last year 1 3.3 3.3 63.3

Gather info about program.

See faculty and students that

I haven't seen in a while.

1 3.3 3.3 66.7

Had a good time 1 3.3 3.3 70.0

Hearing from other students

and learning about their

experiences; It helps to hear

others and how they dealt

with situations

1 3.3 3.3 73.3

Learning about the next

steps in the program.

1 3.3 3.3 76.7

Meeting students; Learning

about the process

1 3.3 3.3 80.0

Networking and seeing other

students/staff

1 3.3 3.3 83.3

Real helpful. Thank you! 1 3.3 3.3 86.7

Talking to other students and

faculty.

1 3.3 3.3 90.0

Thank you for all of your

hard work!!

1 3.3 3.3 93.3

Very Beneficial! 1 3.3 3.3 96.7

Very informative regarding

portfolio process

1 3.3 3.3 100.0

Total 30 100.0 100.0

27


Suggestions for Improvement


Cumulative

Percent

Valid 26 86.7 86.7 86.7

I enjoyed the seminar.

Excellent options for

seminars to at attend. Great

Job!

1 3.3 3.3 90.0

Info from other sessions

would be helpful

1 3.3 3.3 93.3

Please make sure everyone

is informed about the date

1 3.3 3.3 96.7

Shorten 1 3.3 3.3 100.0

Total 30 100.0 100.0

28

Appendix G: Individual Session Evaluations

Distribution Table of Session Attendance

Session Title Frequency Percent Valid Percent

Cumulative

Percent

Valid Higher Ed Policy Updates 8 6.1 6.1 6.1

Current Issues: C&I w/

Meisel

4 3.1 3.1 9.2

IRB 5 3.8 3.8 13.0

Crowdsourcing Research 12 9.2 9.2 22.1

Current Issues: Higher Ed &

K-12

7 5.3 5.3 27.5

3-D Printing Tour 5 3.8 3.8 31.3

K-12 Policy Updates &

Implications

10 7.6 7.6 38.9

Current Issues:C&I

w/Hammond

7 5.3 5.3 44.3

Intro to Doc Studies 9 6.9 6.9 51.1

Student Panel 8 6.1 6.1 57.3

Survey Monkey 8 6.1 6.1 63.4

Makey-Makey 6 4.6 4.6 67.9

Library Skills 9 6.9 6.9 74.8

Career Development after

Dissertations

5 3.8 3.8 78.6

Ed Tech 8 6.1 6.1 84.7

Reflectons on the

Dissertaton Process

5 3.8 3.8 88.5

Reflections on the Portfolio

Process

15 11.5 11.5 100.0

Total 131 100.0 100.0

29

Appendix G: Individual Session Evaluations (continued)

Prior Expertise


Cumulative

Percent

Valid Basic 30 22.9 25.4 25.4

Basic to Intermediate 17 13.0 14.4 39.8

Intermediate 39 29.8 33.1 72.9

Intermediate to Advanced 19 14.5 16.1 89.0

Advanced 13 9.9 11.0 100.0

Total 118 90.1 100.0


Total 131 100.0

Presentation Organized


Cumulative

Percent



Agree 31 23.7 23.7 30.5

Strongly Agree 91 69.5 69.5 100.0

Total 131 100.0 100.0

Efficient Use of Time


Cumulative

Percent



Agree 27 20.6 20.6 24.4

Strongly Agree 99 75.6 75.6 100.0

Total 131 100.0 100.0

30


Efficient Style & Visual Aids


Cumulative

Percent


4.5 1 .8 .8 4.6


Agree 21 16.0 16.2 26.9

Strongly Agree 95 72.5 73.1 100.0

Total 130 99.2 100.0

Missing System 1 .8

Total 131 100.0

Speaker Knowledgeable


Cumulative

Percent

Valid Somewhat Agree 3 2.3 2.3 2.3

Agree 23 17.6 17.6 19.8

Strongly Agree 105 80.2 80.2 100.0

Total 131 100.0 100.0

Speaker Responsive to Questions


Cumulative

Percent

Valid Somewhat Agree 2 1.5 1.5 1.5

Agree 17 13.0 13.0 14.5

Strongly Agree 112 85.5 85.5 100.0

Total 131 100.0 100.0

31


Would Recommend Speaker


Cumulative

Percent



Agree 16 12.2 12.2 17.6

Strongly Agree 108 82.4 82.4 100.0

Total 131 100.0 100.0

Session Contributed to Overall My Knowledge


Cumulative

Percent



Agree 19 14.5 14.6 22.3

Strongly Agree 101 77.1 77.7 100.0

Total 130 99.2 100.0

Missing System 1 .8

Total 131 100.0

32


Areas of Growth


Cumulative

Percent

Valid 67 51.1 51.1 51.1

#edtech 1 .8 .8 51.9

7 1 .8 .8 52.7

7Great hand-outs, Great

survey on Board Members

1 .8 .8 53.4

A fantastic model for me to

know how to organize and

structure my portfolio

1 .8 .8 54.2

A foundation of basic info.

As I had never heard of this

before.

1 .8 .8 55.0

Background about the

common core standards

history was increased.

1 .8 .8 55.7

benefits of social media 1 .8 .8 56.5

Beth is great! 1 .8 .8 57.3

better idea of how to

approach IRB approval

1 .8 .8 58.0

CV development 1 .8 .8 58.8

Educational policy reform 1 .8 .8 59.5

Endnote 1 .8 .8 60.3

Examples were very helpful.

Makes this process less

intimidating.

1 .8 .8 61.1

Excellent 1 .8 .8 61.8

exposure to different

technology,

1 .8 .8 62.6

Found the topics comforting

to my current feelings in the

process of the doctoral

program

1 .8 .8 63.4

33

Gave me more knowledge to

where Ohio might be

headed. We need to turn

the triangle upside down too!

1 .8 .8 64.1

Gave me more to think about

when I start my study.

1 .8 .8 64.9

Good information o the CV.

Good handouts.

1 .8 .8 65.6

Great (possible) research

technique

1 .8 .8 66.4

Great discussions and

thoughtful presenters on

pitfalls and shortcomings in

crowdsourcing.

1 .8 .8 67.2

Great info 1 .8 .8 67.9

Great workshop! 1 .8 .8 68.7

Helped me see possibilities

of 3D printing

1 .8 .8 69.5

I had never seen this before.

Very new technology to me.

1 .8 .8 70.2

I have limited knowledge. I

find this interesting and

wanted to learn more

1 .8 .8 71.0

I knew nothing beforehand

about the policies and how

they impact West Virginia

students. He needs to come

back for future seminars.

1 .8 .8 71.8

I learned a great deal about

crowdsourcing and issues

related to application.

1 .8 .8 72.5

I learned more about the

chair process.

1 .8 .8 73.3

I loved hearing what people

had to say. I am very

interested in this area.

1 .8 .8 74.0

I understand more about C&I 1 .8 .8 74.8

34

I understood the portfolio a

little better

1 .8 .8 75.6

I want one 1 .8 .8 76.3

IDS and EZ Borrow 1 .8 .8 77.1

Importance of differentiation

in teaching methods and

student work.

1 .8 .8 77.9

Improved knowledge in

standards forming and 2510

1 .8 .8 78.6

Indifferent, because the

program seems to be in the

infancy stages, therefore, I

am unsure about it's validity.

1 .8 .8 79.4

Inductive teaching - I learned

more about the topic of

inductive teaching.

1 .8 .8 80.2

Information regarding policy

in higher education in West

Virginia.

1 .8 .8 80.9

Interesting discussion

regarding integrating social

media into the classroom.

1 .8 .8 81.7

Knowledge of WV HE Policy 1 .8 .8 82.4

Learned about many helpful

resources

1 .8 .8 83.2

learned more about the topic 1 .8 .8 84.0

Learning about new tools.

Little bits.

1 .8 .8 84.7

Learning about policies and

where the WVDE is going in

the future to empower

teachers and increase

student learning. The

session had a powerful

message.

1 .8 .8 85.5

35

Learning little tricks how to

make makey makey more

user friendly: bracelet to

gmuadring, use of washers,

etc.

1 .8 .8 86.3

loved this! Great to get to

see how it works & play a

little with it.

1 .8 .8 87.0

Most fun I had all day :-) 1 .8 .8 87.8

New knowledge about

electronic portals to use and

the process. Helpful tips.

1 .8 .8 88.5

None 1 .8 .8 89.3

PortaPortal 1 .8 .8 90.1

Preparing ahead of time is

the key to having a smooth

process in writing the paper.

1 .8 .8 90.8

Provided very good

information for preparing me

for portfolio

1 .8 .8 91.6

Questions answered of

artifacts process

1 .8 .8 92.4

Really interesting. Re-

emphasized a lot of info we

learned in CI 707.

1 .8 .8 93.1

Received much needed

examples concerning

portfolio.

1 .8 .8 93.9

stronger knowledge base of

IRB process

1 .8 .8 94.7

Thank you for the personal

growth experiences you

shared w/ us!

1 .8 .8 95.4

This answered so many

questions!

1 .8 .8 96.2

Understanding of data

sources

1 .8 .8 96.9

36

Understanding of

requirements within program

1 .8 .8 97.7

Understanding

comprehensively HEPC

organization and the status

of state financially. Future

policy decision at institutions

1 .8 .8 98.5

Very good connection to

classroom use.

1 .8 .8 99.2

Very great material! 1 .8 .8 100.0

Total 131 100.0 100.0

37


Recommendations to Speaker


Cumulative

Percent

Valid 98 74.8 74.8 74.8

Deb & Beth are great! 1 .8 .8 75.6

Excellent! 1 .8 .8 76.3

Great Job! 1 .8 .8 77.1

Great job!! Very informative 1 .8 .8 77.9

Great session - visual aids

always help me :-)

1 .8 .8 78.6

Great! 1 .8 .8 79.4

longer session 1 .8 .8 80.2

More info as a requestor. 1 .8 .8 80.9

more time per session 1 .8 .8 81.7

N/A 2 1.5 1.5 83.2

Nice job! Thanks! 1 .8 .8 84.0

None 3 2.3 2.3 86.3

None-Thank you for the

information.

2 1.5 1.5 87.8

None-Thank you so much. I

really needed this

information. It was so

helpful.

1 .8 .8 88.5

None-wonderful, engaging

presentation. I love your

passion for teaching!

1 .8 .8 89.3

None; perfect 1 .8 .8 90.1

None! 1 .8 .8 90.8

Outstanding and

inspirational!

1 .8 .8 91.6

38

Overall, very well run.

Would have liked more time

at the end for discussion.

1 .8 .8 92.4

Show us on the website

where to get forms, etc.

1 .8 .8 93.1

This was a great session &

should be included each

year - it should be suggested

as "idea topic starters" for a

study

1 .8 .8 93.9

Very broad topic for the

allotted time. I would only

discuss 1 issue; rather than

a few….

1 .8 .8 94.7

Very enjoyable and

informative

1 .8 .8 95.4

very good~ 1 .8 .8 96.2

Very interesting. Would like

more discussion although it

picked up at the end. Best

of luck with these initiatives.

1 .8 .8 96.9

Visuals not interesting (black

and white PowerPoint)

1 .8 .8 97.7

Well done - thank you! 1 .8 .8 98.5

Well done! Thank you! 1 .8 .8 99.2

Yes-Extremely Helpful! 1 .8 .8 100.0

Total 131 100.0 100.0

39


Case Processing Summary

Cases

Included Excluded Total

N Percent N Percent N Percent

Prior Expertise * Session

Title

118 90.1% 13 9.9% 131 100.0%

Presentation Organized *

Session Title

131 100.0% 0 0.0% 131 100.0%

Efficient Use of Time *

Session Title

131 100.0% 0 0.0% 131 100.0%

Efficient Style & Visual Aids

* Session Title

130 99.2% 1 0.8% 131 100.0%

Speaker Knowledgeable *

Session Title

131 100.0% 0 0.0% 131 100.0%

Speaker Responsive to

Questions * Session Title

131 100.0% 0 0.0% 131 100.0%

Would Recommend Speaker

* Session Title

131 100.0% 0 0.0% 131 100.0%

Session Contributed to

Overall My Knowledge *

Session Title

130 99.2% 1 0.8% 131 100.0%

Areas of Growth * Session

Title

131 100.0% 0 0.0% 131 100.0%

Recommendations to

Speaker * Session Title

131 100.0% 0 0.0% 131 100.0%

40


Session Title

Prior

Expertise

Presentation

Organized

Efficient

Use of

Time

Efficient Style

& Visual Aids

Speaker

Knowledgeable

Would

Recommend

Speaker

Session

Contributed to

Overall My

Knowledge

Higher Ed Policy

Updates

Mean 3.71 6.13 6.63 6.125 6.88 6.75 6.75

N 7 8 8 8 8 8 8

Std. Deviation 1.496 .835 .518 1.1260 .354 .463 .463

Variance 2.238 .696 .268 1.268 .125 .214 .214

Current Issues: C&I

w/ Meisel

Mean 3.33 7.00 7.00 7.000 7.00 7.00 7.00

N 3 4 4 4 4 4 4

Std. Deviation 2.082 .000 .000 .0000 .000 .000 .000

Variance 4.333 .000 .000 .000 .000 .000 .000

IRB Mean 2.80 6.60 6.80 6.750 7.00 7.00 7.00

N 5 5 5 4 5 5 5

Std. Deviation .447 .894 .447 .5000 .000 .000 .000

Variance .200 .800 .200 .250 .000 .000 .000

Crowdsourcing

Research

Mean 1.67 6.67 6.75 6.000 6.50 6.67 6.58

N 12 12 12 12 12 12 12

Std. Deviation .888 .492 .452 1.2060 .522 .651 .900

Variance .788 .242 .205 1.455 .273 .424 .811

Current Issues:

Higher Ed & K-12

Mean 4.33 6.29 6.43 6.571 6.57 6.57 6.00

N 6 7 7 7 7 7 7

Std. Deviation .816 .951 .787 .7868 .535 .787 1.291

Variance .667 .905 .619 .619 .286 .619 1.667

41

3-D Printing Tour Mean 1.80 6.60 6.60 6.800 6.40 6.60 6.80

N 5 5 5 5 5 5 5

Std. Deviation 1.304 .548 .548 .4472 .548 .548 .447

Variance 1.700 .300 .300 .200 .300 .300 .200

K-12 Policy Updates

& Implications

Mean 3.13 6.80 6.80 6.700 6.90 6.90 6.90

N 8 10 10 10 10 10 10

Std. Deviation .991 .422 .422 .4830 .316 .316 .316

Variance .982 .178 .178 .233 .100 .100 .100

Current Issues: C&I

w/Hammond

Mean 3.43 6.71 6.71 6.857 6.86 6.86 6.86

N 7 7 7 7 7 7 7

Std. Deviation .535 .488 .488 .3780 .378 .378 .378

Variance .286 .238 .238 .143 .143 .143 .143

Intro to Doc Studies Mean 2.11 6.78 6.89 6.667 6.89 6.89 6.89

N 9 9 9 9 9 9 9

Std. Deviation 1.054 .441 .333 .7071 .333 .333 .333

Variance 1.111 .194 .111 .500 .111 .111 .111

Student Panel Mean 2.57 6.25 6.25 6.313 6.75 6.63 6.25

N 7 8 8 8 8 8 8

Std. Deviation 1.618 1.389 1.165 1.2800 .707 1.061 1.165

Variance 2.619 1.929 1.357 1.638 .500 1.125 1.357

Survey Monkey Mean 2.88 6.63 6.75 6.625 6.88 6.75 6.50

N 8 8 8 8 8 8 8

Std. Deviation 1.458 .518 .463 .5175 .354 .463 1.069

Variance 2.125 .268 .214 .268 .125 .214 1.143

Makey-Makey Mean 2.00 6.83 7.00 7.000 7.00 7.00 7.00

N 4 6 6 6 6 6 6

Std. Deviation 1.155 .408 .000 .0000 .000 .000 .000

42

Variance 1.333 .167 .000 .000 .000 .000 .000

Library Skills Mean 2.75 6.78 6.89 6.889 7.00 7.00 7.00

N 8 9 9 9 9 9 9

Std. Deviation 1.165 .441 .333 .3333 .000 .000 .000

Variance 1.357 .194 .111 .111 .000 .000 .000

Career Development

after Dissertations

Mean 3.50 6.40 6.40 6.200 6.60 6.40 6.20

N 4 5 5 5 5 5 5

Std. Deviation 1.000 .548 .548 .8367 .548 .894 .447

Variance 1.000 .300 .300 .700 .300 .800 .200

Ed Tech Mean 2.14 6.38 6.25 6.375 6.38 6.25 6.25

N 7 8 8 8 8 8 8

Std. Deviation 1.464 .916 1.165 .9161 .916 1.165 1.165

Variance 2.143 .839 1.357 .839 .839 1.357 1.357

Reflectons on the

Dissertaton Process

Mean 3.25 5.80 6.60 6.200 6.60 6.20 6.40

N 4 5 5 5 5 5 5

Std. Deviation 1.708 1.643 .548 1.3038 .548 1.304 .894

Variance 2.917 2.700 .300 1.700 .300 1.700 .800

Reflections on the

Portfolio Process

Mean 2.57 6.93 6.93 6.933 6.93 6.93 6.79

N 14 15 15 15 15 15 14

Std. Deviation 1.016 .258 .258 .2582 .258 .258 .426

Variance 1.033 .067 .067 .067 .067 .067 .181

Total Mean 2.73 6.60 6.70 6.581 6.78 6.75 6.66

N 118 131 131 130 131 131 130

Std. Deviation 1.305 .731 .591 .7944 .469 .624 .732

Variance 1.704 .535 .349 .631 .220 .390 .536

edf 711: survey research...seminar evaluations have been lacking in providing specific feedback to...

Documents